OpenBay

GETTING STARTED

The installation steps are tested on virtual machine managed by VirtualBox. The virtual PC was configured with 2 network cards, the first one attached to NAT, the second one - Host-only adapter. You'll need the IP of host only-adapter to configure host machine, where you're intending to develop and test openbay. If the second adapter wasn't configured automatically - edit /etc/network/interfaces file by adding

auto eth1
iface eth1 inet static
address 192.168.56.101
netmask 255.255.255.0

Then restart.

By default VirtualBox Host-only network address is 192.168.56.*. If not - refer to Linux networking and VirtualBox documentation.

Operating system

Install Ubuntu 14.04 32-bit minimal from here on your virtual pc. To reduce system overhead, do not install packets which are not necessary for you. In this case, to control installation, only Basic Server and OpenSSH Server are chosen. During installation create user named, for example, openpirate. The installation is quite trivial, for details visit official site.

Install some useful and necessary software

$ sudo apt-get update
$ sudo apt-get install git curl

This will install curl and git to your system.

Checkout working copy

If you want to checkout to different path - fix path in configs. TODO: Change repo name after merge pull-request

$ mkdir ~/www/
$ cd ~/www/
$ git clone https://github.com/profezz/openbay.git --branch manual

LNMP stack (nginx + apache + php)

For using OpenBay you need LNMP stack. The installation of nginx, mysql and PHP-fpm will be described in this section. If you want to use other stack (for example LAMP) refer to official documentation.

Nginx

Nginx is a high performance scalable web-server. Install it by typing

$ sudo apt-get install nginx

Change server run user to openpirate in /etc/nginx/nginx.conf.

Copy nginx config file from working copy to /etc/nginx/sites-available/opb. Edit paths if necessary.

sudo cp ~/www/openbay/environments/dev/conf/nginx.conf /etc/nginx/sites-available/opb.conf

Create symlink in /etc/nginx/sites-enabled/

$ sudo ln -s /etc/nginx/sites-available/opb.conf /etc/nginx/sites-enabled/001-opb.conf

Make logs dir

mkdir ~/www/logs

Now restart nginx.

$ sudo service nginx restart

Mysql

You will need both mysql-server and mysql-client to make Openbay work.

$ sudo apt-get install mysql-server-5.5 mysql-client

Root password will be prompted during installation.

PHP

This manual describes how to install php-fpm. If you want to use FastCGI refer to PHP documentation.

Install PHP with necessary modules

$ sudo apt-get install php5-fpm php5-memcached php-pear php5-redis php5-dev php5-mysql php5-mcrypt php5-intl

Change FPM user and group and FPM socket owner and group from www-data to openpirate in /etc/php5/fpm/pool.d/www.conf

Also, you'll have to install igbinary extension manually.

$ sudo pecl install -Z igbinary

And create and link necessary files:

• create /etc/php5/mods-available/igbinary.ini with following content:

extension=igbinary.so

• Create symlinks for CLI and FPM config files

$ sudo ln -s /etc/php5/mods-available/igbinary.ini /etc/php5/cli/conf.d/10-igbinary.ini
$ sudo ln -s /etc/php5/mods-available/igbinary.ini /etc/php5/fpm/conf.d/10-igbinary.ini
$ sudo ln -s /etc/php5/mods-available/mcrypt.ini /etc/php5/cli/conf.d/10-mcrypt.ini
$ sudo ln -s /etc/php5/mods-available/mcrypt.ini /etc/php5/fpm/conf.d/10-mcrypt.ini

Now, restart FPM

$ sudo service php5-fpm restart

Sphinx

Sphinx is powerful and flexible search engine used in Openbay.

Download sphinxsearch from official site

$ wget http://sphinxsearch.com/files/sphinxsearch_2.2.7-release-0ubuntu12~precise_i386.deb

Install sphinx

$ sudo apt-get install libodbc1 libpq5
$ sudo dpkg -i sphinxsearch_2.2.7-release-0ubuntu12~precise_i386.deb

Stop the daemon

$ sudo searchd --stop

Move sphinx config to /etc/sphinx

$ sudo cp ~/www/openbay/environments/dev/conf/sphinx.conf /etc/sphinxsearch/sphinx.conf

Redis

The server user for caching.

$ sudo apt-get install redis-server

composer

Now install composer.

$ curl -s http://getcomposer.org/installer | php
$ sudo mv composer.phar /usr/local/bin/composer

Initializing project

Go to sources folder

$ cd ~/www/openbay/

You need to install composer bower plugin

$ composer global require "fxp/composer-asset-plugin:1.0.0"

And get vendor packages

$ composer update

Due to bower assets it can take long time.

Also, composer can require your github credentials.

Now create database

$ mysql -uroot -p
mysql> create database opb default character set utf8;
mysql> grant all on opb.* to opb@'%' identified by 'opb';

Initialize project by running

$ cd ~/www/openbay
$ php init

Fill salts and reCaptcha key in ~/www/openbay/frontend/config/main-local.php

Apply migrations

$ ./yii migrate
$ ./yii migrate --migrationPath=@frontend/modules/comment/migrations
$ ./yii migrate --migrationPath=@frontend/modules/complain/migrations
$ ./yii migrate --migrationPath=@frontend/modules/rating/migrations

Now you can rebuild sphinx indices

$ sudo indexer --all

And run sphinx

$ sudo killall searchd
$ sudo searchd

Due to sphinx config it seems that only delta index should be rotated on data change

$ indexer --rotate npbtorrents_delta

And now Openbay is installed. To test it you need some actions on host machine. Append the following line to /etc/hostsfile:

192.168.56.101 opb.virt

And you can test your developer copy of Openbay by typing http://opb.virt in your browser.

from https://github.com/isohuntto/openbay

--------------

Locally index and search database dumps from https://openbay.isohunt.to using the bleve text indexing library.

TPB Search

Locally index and search database dumps from The Open Bay using the bleve text indexing library.

Based on the example applications beer-search and bleve-explorer

 

Requirements

You need to download an openbay-db-dump

It should have the following format when gunzipped:

"Name of the first file"|54321|f2b2c2e4a786d3924b8922454772d784118e6421|8|music|0|0
"Name of the second file"|9876|27a12d50782e1412bcdec126224da29afb23c293|1|movies|0|0

You also need to install LevelDB

brew install leveldb

Compilation

git clone git@github.com:peterhellberg/tpb-search.git
cd tpb-search
make deps
make

Usage

Usage of ./tpb-search:
  -b=800: batch size for indexing
  -d="torrents_mini.csv.gz": the openbay-db-dump to use
  -h="localhost": http listen host
  -i="tpb.index": index path
  -p=1337: http listen port

Building an index of the first 10000 rows in the dump:

$ GOMAXPROCS=4 make && ./tpb-search -l 10000
rm -f tpb-search assets.go
go-bindata -o assets.go -prefix "assets/" assets/...
go build -tags leveldb -ldflags "-X main.buildCommit `git rev-parse --short HEAD`" -o tpb-search .
2014/12/21 04:45:18 Creating new index...
2014/12/21 04:45:18 Listening on http://localhost:1337
2014/12/21 04:45:18 Indexing...
2014/12/21 04:45:19 Indexed 1000 documents in 1.16s (average 1.16ms/doc)
2014/12/21 04:45:19 Indexed 2000 documents in 1.47s (average 0.73ms/doc)
2014/12/21 04:45:20 Indexed 3000 documents in 1.82s (average 0.61ms/doc)
2014/12/21 04:45:20 Indexed 4000 documents in 2.50s (average 0.63ms/doc)
2014/12/21 04:45:21 Indexed 5000 documents in 2.91s (average 0.58ms/doc)
2014/12/21 04:45:21 Indexed 6000 documents in 3.33s (average 0.56ms/doc)
2014/12/21 04:45:22 Indexed 7000 documents in 3.85s (average 0.55ms/doc)
2014/12/21 04:45:22 Indexed 8000 documents in 4.79s (average 0.60ms/doc)
2014/12/21 04:45:23 Indexed 9000 documents in 5.26s (average 0.58ms/doc)
2014/12/21 04:45:23 Indexed 10000 documents in 5.74s (average 0.57ms/doc)
2014/12/21 04:45:24 Finished indexing 10000 documents in 6.04s (average 0.60ms/doc)
2014/12/21 04:45:24 Still listening on http://localhost:1337

Third party packages

• bleve
• bleve/http
• gorilla/mux
• levigo

from https://github.com/peterhellberg/tpb-search

--------------

An open source distributed implementation of the open bay.

The Distributed Bay is a peer-to-peer rewrite of the Open Bay/Pirate Bay. It uses a completely distributed protocol to share entries between numerous nodes, this makes it nearly impossible to take down by seizure of servers. Each node contains a complete copy and any changes are propagate through the network.

Running with Docker

On Ubuntu 14.10:

sudo apt-get install docker.io
sudo docker run thedistributedbay/thedistributedbay

Docker is a container deploying environment that provides automated container images from git.

The Distributed Bay Registry page: https://registry.hub.docker.com/u/thedistributedbay/thedistributedbay/

Running Manually

Assuming you have Go and the GOPATH correctly configured all you have to do is run:

go get github.com/TheDistributedBay/TheDistributedBay
go install github.com/TheDistributedBay/TheDistributedBay

$GOPATH/bin/TheDistributedBay

Development

Backend/Core

The Distributed Bay is primarily written in Go.

To get the source code install Go, configure the GOPATH and then run:

go get github.com/TheDistributedBay/TheDistributedBay

The source code will be available in $GOPATH/src/github.com/TheDistributedBay/TheDistributedBay.

Frontend

The frontend is written in AngularJS and located in frontend/angular/app.

The compiled frontend assets are checked into Git. This is done so the backend developers don't need to worry about the frontend and to make deployments easier.

To edit the frontend you'll need to first install Node.js or io.js.

You'll also need to install Ruby and the gem compass.

Once done, navigate into the frontend/angular folder and install the dependencies.

cd frontend/angular
npm install
npm install -g bower grunt-cli
bower install
gem install compass

Grunt is used to handle compilation of the frontend. To tell Grunt to automatically recompile the SCSS files run:

grunt serve

You'll also need to tell the backend to serve the development assets from frontend/angular/app instead of the production ones in frontend/angular/dist. You can do this by running:

go run main -devassets=true

from https://github.com/TheDistributedBay/TheDistributedBay