Total Pageviews

Sunday, 3 January 2016

OpenBay

GETTING STARTED

The installation steps are tested on virtual machine managed by VirtualBox. The virtual PC was configured with 2 network cards, the first one attached to NAT, the second one - Host-only adapter. You'll need the IP of host only-adapter to configure host machine, where you're intending to develop and test openbay. If the second adapter wasn't configured automatically - edit /etc/network/interfaces file by adding
auto eth1
iface eth1 inet static
address 192.168.56.101
netmask 255.255.255.0
Then restart.
By default VirtualBox Host-only network address is 192.168.56.*. If not - refer to Linux networking and VirtualBox documentation.

Operating system

Install Ubuntu 14.04 32-bit minimal from here on your virtual pc. To reduce system overhead, do not install packets which are not necessary for you. In this case, to control installation, only Basic Server and OpenSSH Server are chosen. During installation create user named, for example, openpirate. The installation is quite trivial, for details visit official site.

Install some useful and necessary software

$ sudo apt-get update
$ sudo apt-get install git curl
This will install curl and git to your system.

Checkout working copy

If you want to checkout to different path - fix path in configs. TODO: Change repo name after merge pull-request
$ mkdir ~/www/
$ cd ~/www/
$ git clone https://github.com/profezz/openbay.git --branch manual

LNMP stack (nginx + apache + php)

For using OpenBay you need LNMP stack. The installation of nginx, mysql and PHP-fpm will be described in this section. If you want to use other stack (for example LAMP) refer to official documentation.
Nginx
Nginx is a high performance scalable web-server. Install it by typing
$ sudo apt-get install nginx
Change server run user to openpirate in /etc/nginx/nginx.conf.
Copy nginx config file from working copy to /etc/nginx/sites-available/opb. Edit paths if necessary.
sudo cp ~/www/openbay/environments/dev/conf/nginx.conf /etc/nginx/sites-available/opb.conf
Create symlink in /etc/nginx/sites-enabled/
$ sudo ln -s /etc/nginx/sites-available/opb.conf /etc/nginx/sites-enabled/001-opb.conf
Make logs dir
mkdir ~/www/logs
Now restart nginx.
$ sudo service nginx restart
Mysql
You will need both mysql-server and mysql-client to make Openbay work.
$ sudo apt-get install mysql-server-5.5 mysql-client
Root password will be prompted during installation.
PHP
This manual describes how to install php-fpm. If you want to use FastCGI refer to PHP documentation.
Install PHP with necessary modules
$ sudo apt-get install php5-fpm php5-memcached php-pear php5-redis php5-dev php5-mysql php5-mcrypt php5-intl
Change FPM user and group and FPM socket owner and group from www-data to openpirate in /etc/php5/fpm/pool.d/www.conf
Also, you'll have to install igbinary extension manually.
$ sudo pecl install -Z igbinary
And create and link necessary files:
  • create /etc/php5/mods-available/igbinary.ini with following content:
extension=igbinary.so
  • Create symlinks for CLI and FPM config files
$ sudo ln -s /etc/php5/mods-available/igbinary.ini /etc/php5/cli/conf.d/10-igbinary.ini
$ sudo ln -s /etc/php5/mods-available/igbinary.ini /etc/php5/fpm/conf.d/10-igbinary.ini
$ sudo ln -s /etc/php5/mods-available/mcrypt.ini /etc/php5/cli/conf.d/10-mcrypt.ini
$ sudo ln -s /etc/php5/mods-available/mcrypt.ini /etc/php5/fpm/conf.d/10-mcrypt.ini
Now, restart FPM
$ sudo service php5-fpm restart

Sphinx

Sphinx is powerful and flexible search engine used in Openbay.
Download sphinxsearch from official site
$ wget http://sphinxsearch.com/files/sphinxsearch_2.2.7-release-0ubuntu12~precise_i386.deb
Install sphinx
$ sudo apt-get install libodbc1 libpq5
$ sudo dpkg -i sphinxsearch_2.2.7-release-0ubuntu12~precise_i386.deb
Stop the daemon
$ sudo searchd --stop
Move sphinx config to /etc/sphinx
$ sudo cp ~/www/openbay/environments/dev/conf/sphinx.conf /etc/sphinxsearch/sphinx.conf

Redis

The server user for caching.
$ sudo apt-get install redis-server

composer

Now install composer.
$ curl -s http://getcomposer.org/installer | php
$ sudo mv composer.phar /usr/local/bin/composer

Initializing project

Go to sources folder
$ cd ~/www/openbay/
You need to install composer bower plugin
$ composer global require "fxp/composer-asset-plugin:1.0.0"
And get vendor packages
$ composer update
Due to bower assets it can take long time.
Also, composer can require your github credentials.
Now create database
$ mysql -uroot -p
mysql> create database opb default character set utf8;
mysql> grant all on opb.* to opb@'%' identified by 'opb';
Initialize project by running
$ cd ~/www/openbay
$ php init
Fill salts and reCaptcha key in ~/www/openbay/frontend/config/main-local.php
Apply migrations
$ ./yii migrate
$ ./yii migrate --migrationPath=@frontend/modules/comment/migrations
$ ./yii migrate --migrationPath=@frontend/modules/complain/migrations
$ ./yii migrate --migrationPath=@frontend/modules/rating/migrations
Now you can rebuild sphinx indices
$ sudo indexer --all
And run sphinx
$ sudo killall searchd
$ sudo searchd
Due to sphinx config it seems that only delta index should be rotated on data change
$ indexer --rotate npbtorrents_delta
And now Openbay is installed. To test it you need some actions on host machine. Append the following line to /etc/hostsfile:
192.168.56.101 opb.virt
And you can test your developer copy of Openbay by typing http://opb.virt in your browser.

from https://github.com/isohuntto/openbay
--------------
Locally index and search database dumps from https://openbay.isohunt.to using the bleve text indexing library.

TPB Search

License MIT
Locally index and search database dumps from The Open Bay using the bleve text indexing library.
Based on the example applications beer-search and bleve-explorer
Search Prefix

Requirements

You need to download an openbay-db-dump
It should have the following format when gunzipped:
"Name of the first file"|54321|f2b2c2e4a786d3924b8922454772d784118e6421|8|music|0|0
"Name of the second file"|9876|27a12d50782e1412bcdec126224da29afb23c293|1|movies|0|0
You also need to install LevelDB
brew install leveldb

Compilation

git clone git@github.com:peterhellberg/tpb-search.git
cd tpb-search
make deps
make

Usage

Usage of ./tpb-search:
  -b=800: batch size for indexing
  -d="torrents_mini.csv.gz": the openbay-db-dump to use
  -h="localhost": http listen host
  -i="tpb.index": index path
  -p=1337: http listen port

Building an index of the first 10000 rows in the dump:

$ GOMAXPROCS=4 make && ./tpb-search -l 10000
rm -f tpb-search assets.go
go-bindata -o assets.go -prefix "assets/" assets/...
go build -tags leveldb -ldflags "-X main.buildCommit `git rev-parse --short HEAD`" -o tpb-search .
2014/12/21 04:45:18 Creating new index...
2014/12/21 04:45:18 Listening on http://localhost:1337
2014/12/21 04:45:18 Indexing...
2014/12/21 04:45:19 Indexed 1000 documents in 1.16s (average 1.16ms/doc)
2014/12/21 04:45:19 Indexed 2000 documents in 1.47s (average 0.73ms/doc)
2014/12/21 04:45:20 Indexed 3000 documents in 1.82s (average 0.61ms/doc)
2014/12/21 04:45:20 Indexed 4000 documents in 2.50s (average 0.63ms/doc)
2014/12/21 04:45:21 Indexed 5000 documents in 2.91s (average 0.58ms/doc)
2014/12/21 04:45:21 Indexed 6000 documents in 3.33s (average 0.56ms/doc)
2014/12/21 04:45:22 Indexed 7000 documents in 3.85s (average 0.55ms/doc)
2014/12/21 04:45:22 Indexed 8000 documents in 4.79s (average 0.60ms/doc)
2014/12/21 04:45:23 Indexed 9000 documents in 5.26s (average 0.58ms/doc)
2014/12/21 04:45:23 Indexed 10000 documents in 5.74s (average 0.57ms/doc)
2014/12/21 04:45:24 Finished indexing 10000 documents in 6.04s (average 0.60ms/doc)
2014/12/21 04:45:24 Still listening on http://localhost:1337

Third party packages

from https://github.com/peterhellberg/tpb-search
--------------

An open source distributed implementation of the open bay.

The Distributed Bay is a peer-to-peer rewrite of the Open Bay/Pirate Bay. It uses a completely distributed protocol to share entries between numerous nodes, this makes it nearly impossible to take down by seizure of servers. Each node contains a complete copy and any changes are propagate through the network.

Running with Docker

On Ubuntu 14.10:
sudo apt-get install docker.io
sudo docker run thedistributedbay/thedistributedbay
Docker is a container deploying environment that provides automated container images from git.

Running Manually

Assuming you have Go and the GOPATH correctly configured all you have to do is run:
go get github.com/TheDistributedBay/TheDistributedBay
go install github.com/TheDistributedBay/TheDistributedBay

$GOPATH/bin/TheDistributedBay

Development

Backend/Core

The Distributed Bay is primarily written in Go.
GoDoc
To get the source code install Go, configure the GOPATH and then run:
go get github.com/TheDistributedBay/TheDistributedBay
The source code will be available in $GOPATH/src/github.com/TheDistributedBay/TheDistributedBay.

Frontend

The frontend is written in AngularJS and located in frontend/angular/app.
The compiled frontend assets are checked into Git. This is done so the backend developers don't need to worry about the frontend and to make deployments easier.
To edit the frontend you'll need to first install Node.js or io.js.
You'll also need to install Ruby and the gem compass.
Once done, navigate into the frontend/angular folder and install the dependencies.
cd frontend/angular
npm install
npm install -g bower grunt-cli
bower install
gem install compass
Grunt is used to handle compilation of the frontend. To tell Grunt to automatically recompile the SCSS files run:
grunt serve
You'll also need to tell the backend to serve the development assets from frontend/angular/app instead of the production ones in frontend/angular/dist. You can do this by running:
go run main -devassets=true
from https://github.com/TheDistributedBay/TheDistributedBay
Posted by at
Labels: , , ,