Total Pageviews

Wednesday, 23 March 2016

FuseCompress-compressed filesystem in userspace

Transparent (using fuse) compressing filesystem。
FuseCompress (compressed filesystem in userspace)
 http://www.miio.net/fusecompress

We are not responsible for any badness that may happen to you during using
this software.

Todo:

Bugs:
 File becames unreadable when it hits a not enough space condition. So
 be careful to really have enough free space on the disk to store all the data.

This is more a feature than a bug:

 In CFile *FileManager::GetUnlocked(const char *name, bool create) FuseCompress
 uses the underlying filesystem inode number. This is a problem when you have
 multiple partitions underneath.

 For example

 /backups/.max (partition 1)
 /backups/.max/alex (partition 2)
 /backups/.max -> /backup/max (fusecompress)

 This would spoil the inode pool and potentially lead to conflicts, especially
 with hard links.

Problems:
 Do NOT compile FuseCompress with g++-4.3.3! Strange threading problems
 (broken locking) were detected (thank you Steve!) when using this version of
 g++.

 There is a memory leak (not too big, user detects it after milions of
 copyied files) in 'file' package, more exactly in it's magic library, see:

 http://mx.gw.com/pipermail/file/2009/000306.html

==13747== 1,356 bytes in 135 blocks are definitely lost in loss record 9 of 12                                                                                             
==13747==    at 0x4C256AE: malloc (in /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so)                                                                               
==13747==    by 0x725503F: __vasprintf_chk (in /lib64/libc-2.9.so)                                                                                                         
==13747==    by 0x7254ED2: __asprintf_chk (in /lib64/libc-2.9.so)                                                                                                          
==13747==    by 0x63C2993: file_printf (in /usr/lib64/libmagic.so.1.0.0)                                                                                                   
==13747==    by 0x63BBDC5: (within /usr/lib64/libmagic.so.1.0.0)                                                                                                           
==13747==    by 0x63BC6A7: file_softmagic (in /usr/lib64/libmagic.so.1.0.0)                                                                                                
==13747==    by 0x63C2B8A: file_buffer (in /usr/lib64/libmagic.so.1.0.0)                                                                                                   
==13747==    by 0x63B6104: magic_buffer (in /usr/lib64/libmagic.so.1.0.0)                                                                                                  
==13747==    by 0x41F9CF: CompressedMagic::isNativelyCompressed(char const*, int) (CompressedMagic.cpp:140)                                                                
==13747==    by 0x42A44F: Compress::write(char const*, unsigned long, long) (Compress.cpp:361)                                                                             
==13747==    by 0x43B99E: Memory::write(bool) (Memory.cpp:203)                                                                                                             
==13747==    by 0x43BC77: Memory::merge(char const*) (Memory.cpp:53)                                                                                                       
==13747==                                                                                                                                                                  

 This leak should be fixed with the newer 'file' package.

Requires:
 boost >= 1.33.1

 libz
 libbz2
 xz-4.999.9beta (lzma library from http://tukaani.org/xz/xz-4.999.9beta.tar.gz)
 liblzo2

Example (with apt-get):

 $ sudo apt-get install libboost-dev libz-dev liblzo2-dev libmagic-dev libfuse-dev -y
 (also install the xz-beta manually, above)
 Note that 32-bit users may need to recompile boost by hand for long file support (see below).

Compile:

Typically you only need to run ./configure && make && sudo make install

 Boost's libraries may be stored in /usr/lib64 on 64bit systems,
 configure script needs to be called with:

 ./configure --with-boost-libdir=/usr/lib64/

 Don't use --enable-debug flag. Build fails during linking with boost
 libraries.

 If you don't have pkg-tool installed, try following:

 FUSE_CFLAGS="-D_GNU_SOURCE -D_REENTRANT -D_POSIX_C_SOURCE=200112L -D_POSIX_SOURCE -D_DEFAULT_SOURCE -D_XOPEN_SOURCE=500 -pedantic -Wno-long-long -Wall -fpermissive -D_FILE_OFFSET_BITS=64 -DFUSE_USE_VERSION=26" FUSE_LIBS="-lfuse" ./configure --with-boost=/usr/include/ --with-boost-libdir=/usr/lib64

 If your boost library was compiled for 32bit system without support for long files (bigger than 2GiB)
 configure script prints this error message:
  Boost library compiled for 32bit architecture without long file support!

 It means that any program using this library is limited to work with files smaller than 2GiB.
 You can either ignore this problem by passing --enable-crippled_boost to FuseCompress' configure script
 or compile boost with long file support. In both cases, try to report this problem to authors of your
 Linux distribution.

 There are switches you can use to build boost library with long file support:

 ./bjam "define=_FILE_OFFSET_BITS=64" release --toolset=gcc --build-type=minimal --layout=system --libdir=/usr/lib/ --prefix=/usr
 sudo ./bjam "define=_FILE_OFFSET_BITS=64" release --toolset=gcc --build-type=minimal --layout=system --libdir=/usr/lib/ --includedir=/usr/include/boost --prefix=/usr install

Usage:

 See man pages for fusecompress and fusecompress_offline.

Author:

Milan Svoboda <milan.svoboda@centrum.cz> (author and project maintainer)

Feel free to contact me with suggestions and bug reports.
 

from https://github.com/tex/fusecompress
 
相关帖子:
http://briteming.blogspot.com/2013/02/fusecompress-liveusb.html
http://briteming.blogspot.com/2013/02/fusecompress.html