Total Pageviews

Sunday, 4 April 2021

On Password Strength

 Note that this post focuses on the ability to determine the strength of a randomly generated password with high precision. While the password l3dz3pp3linRul3z! contains an undisputed fact, it is not an example of a secure or strong password, for obvious reasons. Password security is a larger topic and is not the focus of this one.

I’ve recently been doing a lot of reading and research about best practices for protecting one’s privacy on the Internet, in anticipation of doing some public speaking on the subject. While no stranger to privacy and anonymity tools, I’ve found it interesting and instructive to approach the subject from the perspective of a teacher rather than that of a user. In the process, I’ve done deep dives and covered a lot of territory to make sure that I’m quite comfortable with any of the topics I’ll be presenting, and I’ll also naturally be fielding questions.

During my reading on passwords (a sometimes boring but quite important topic!), one thing that greatly captured my interest was that of password strength. I’d never stopped to think about how a password can be quantitatively measured and reasoned about; a password’s strength can be measured in terms of information entropy, which is defined as the average amount of information produced by a random, or stochastic, source of data. This information is measured in bits (or shannons, named after Claude Shannon, the mathematician credited with the concept of information entropy).

Importantly, only randomly-generated passwords can be calculated with precision; human-generated passwords are rarely random and therefore their strength is very difficult to determine. The algorithm is quite straightforward and easy to understand and is a function of length, complexity and randomness. A random password’s information entropy H is determined by the following equation:

H = log2(NL)

I find it useful to think of the symbol set(s) used for a given password as analogous to its key space. For example, an Arabic numeral password has a key space of 10, the lowercase Latin alphabet has a key space of 26 characters, as does its uppercase counterpart, and the mixed-case Latin alphabet has a key space of 52. It follows, then, that a case insensitive alphanumeric password has a key space of 62 ([a-z][A-Z][0-9]).

So, for instance, if we wanted to see how many entropy bits a single character password would have given an Arabic numeral character set, the answer would be calculated thusly:

H = log2(10) = 3.3219280949

For a single character, case insensitive Latin alphabet:

H = log2(52) = 5.7004397181

Single character, all printable ascii characters except space:

H = log2(94) = 6.5545888517

Diceware* passphrase (one word):

H = log2(7776) = 12.924812504

The Diceware method for generating passphrases is a list of 7776 words, hence calculating the base-2 logarithm of 7776.

As you’d expect, the larger the symbol set (key space), the greater the bit strength for an individual character of a password. Armed with this information, we now can reason about our passwords and calculate their strength with great precision. Depending upon the threat model, a long password may not be needed, especially when employing key stretching. However, generally, the more entropy the better, and so recommendations can vary between 29 bits and 96 bits. I favor a six word Diceware passphrase, so my passphrases clock in at roughly 77.549 entopy bits. Of course, all bets are off when quantum computers are mainstream, which will be much sooner than you think. Time to learn lattice-based cryptography!

* I’m a fan of the Diceware method for generating strong and memorable passphrases. I’ve written two implementations, one in JavaScript and one in Go, and I use it as a plugin in my password manager.

from  https://www.benjamintoll.com/2018/05/30/on-password-strength/

On systemd-nspawn

 I was recently researching container technology (again), as I seem to do every once in a while. I’ve been using chroots for a long time and have written about them a bit. I love the idea of sandboxes, so I’ve always been drawn to the concept and use them when needed.

I’ve been down this road before, and every time I’m hesitant to use the most well-known player in this domain, Docker. Why? Well, almost to a person, the one recommending that I use Docker has never heard of chroot or BSD jails. This is usually indicative that that person has not done their homework and has just adopted Docker. This could be for a multitude of reasons, but none of them are reasonable or smart unless they’ve also weighed the alternatives, such as ones offered natively by various Unices. That may not be fair, but that’s how the cookie crumbles.

A Brief History

Container technologies are old. According to the historical record, they first came into widespread use by the Etruscans before they succumbed to the Romans, at which point the technology was unfortunately lost for centuries.

The story picks back up in the 20th century at Bell Labs, where the idea of a chroot made its way into Version 7 Unix in 1979. Bill Joy, he of vi fame (thanks Bill!), added it to BSD in 1982, and the stage was set for fully-virtualized sandboxed environments termed jails that were released in FreeBSD in 2002. Jails were known to be used in Linux at least by 2002.

Containers, as we know and speak of them today, were first developed (marketed?) by Sun Microsystems, who called them Solaris Containers in 2005, and Linux followed in 2008 with their operating-system level virtualization method, LXC. The latter is the technology that Docker was first built-upon, although it no longer uses it.

By 2013, Linux natively supports the building blocks for containers (cgroups, namespaces) in the kernel (starting with version 3.8), which considerably opens the reach of containers to a broad audience.

So Docker? Unless I’m missing something, they’ve just been successful at marketing something old as something new. And they have a cute mascot.

Creating Containers

In this article, we’re going to look at one particular container technology built into systemd called systemd-nspawn. systemd is an init system that has been widely adopted by many Linux distributions, so the chances are very good that you already have it installed (because you are using Linux, right?). And, if not, it’s just an install away.

systemd has been highly controversial within the Linux community. I’m not going to get into it here, but long story short, many people feel that it doesn’t subscribe to the Unix philosophy. I think they have a good point, and if you’re not using systemd, I don’t know if you need to download the entire package just to be able to use the container utilities of which I’m speaking here. You may be able to get away with just downloading the systemd-container package. I really don’t know, but I suspect not, which would understandably turn some people off.

systemd-nspawn is a utility that ships with systemd, so most Linux users will have it “out-of-the-box” with most standard installations (if not present on your system, simply download the systemd-container package). This is the utility that allows you to spawn containers. The machinectl utility will also be a part of the systemd-container package, and this will enable you to manage your containers via a simple inteface (although everything can be done using systemd-nspawn).

I won’t be talking about Linux containers or Solaris Containers, but they are also popular and exciting container technologies.

There are several steps I took to create my workspace and get containers up and running:

  1. Create an area on disk. We’ll do this by creating a btrfs filesystem mount point where we’ll create our containers.

  2. Download and install a minimal Debian system.

  3. Create and manage our containers.

If you’re downloading an image or tar file from an external source then you can skip the first two steps and go right to step 3.

btrfs

Why btrfs? This is a great layered filesystem that is very full-featured, the most important of which for our purposes is the ability to easily and quickly create snapshots of a btrfs subvolume. These snapshots will be our new containers.

The idea is to make a btrfs filesystem workspace within our filesystem. With this then mounted, we’ll create a subvolume (directory) that will be the base installations from which we’ll stamp out our containers by creating our snapshots (although, we’ll shortly see how machinectl will manage this for us). Weeeeeeeeeeeeeeeeee

I’ll create this in /srv, because, why not?

Note all commands from here on are performed as root!

# Allocate 20GiB space for the filesystem.
fallocate -l 20G containers

# Create the btrfs filesystem.
mkfs.btrfs containers

# Create the mount point and mount it.
mkdir trees
mount -o loop containers trees

# Finally, create a subvolume that we'll use as the install target.
btrfs subvolume create debian_build-vim

debootstrap

The second step to creating a container is to download the packages needed to bootstrap a virtual environment. For this, I’ll use debootstrap, since I run Debian (other distros will have similar tool, such as pacstrap for Arch Linux). This is easy to do. At its simplest:

debootstrap [version] [location]

However, you’ll probably want to set some flags to control the behavior, such as the variant type (the type of download, such as minbase (the default), buildd or fakechroot) and other packages to download along with it:

debootstrap --arch=amd64 --variant=buildd --include=vim,curl,git stretch debian-buildd/ http://deb.debian.org/debian

The notable thing about this command is that we’re downloading packages for a minimal install along with build-essential (specified by --variant=buildd) and the packages for vim, curl and git. Basically, a development environment, which can serve as a base for more specific projects.

You can tell debootstrap to simply download the deb packages without installing them. I particularly like this feature since I’m bandwidth-deprived. So, I went to the library and downloaded several base packages into their respective tar files:

debootstrap --arch=amd64 --variant=minbase --include=vim --make-tarball=stretch-minbase_vim.tgz stretch foo
debootstrap --arch=amd64 --variant=buildd --include=vim,curl,git --make-tarball=stretch-buildd_vim-curl-git.tgz stretch foo
et al.

Note that when using debootstrap to create an archive the command still needs a location, even though we’re not actually installing anything. In this case, I just made an empty directory foo which allows me to run the command.

This will allow me to point the installation at the archive rather than downloading them via HTTP, allowing me to save precious bits. You could also place these archives on a local network drive which would be save not only bandwidth but time.

So, here is what the process was for me to download and install a base container:

debootstrap --arch=amd64 --variant=minbase --include=vim --make-tarball=stretch-minbase_vim.tgz stretch foo
debootstrap --unpack-tarball=/srv/trees/packages/stretch-minbase_vim.tgz stretch debian_base-stretch

That this didn’t need to be a two-step process, but it’s nice to save the bandwidth for subsequent installs. Also, minbase is the default, but I like to be explicit, especially when I’m first using a technology. Here is the one-liner:

debootstrap --arch=amd64 --variant=minbase --include=vim stretch debian_base-stretch http://deb.debian.org/debian

I did that once for all of the different combination minimal package archives that I anticipate needing (at least for stretch), and so moving forward I won’t need to do it again.

I now have all my base package tarballs in the containers tree:

/srv/containers/packages:$ ll
total 315M
-rw------- 1 root root 96M Aug 20 14:33 stretch-buildd_vim-curl-git.tgz
-rw------- 1 root root 91M Aug 20 14:28 stretch-buildd_vim-curl.tgz
-rw------- 1 root root 86M Aug 20 14:31 stretch-buildd_vim.tgz
-rw------- 1 root root 43M Aug 20 14:25 stretch-minbase_vim.tgz

Let’s install one! Weeeeeeeeeeeeeeeeee

debootstrap --unpack-tarball=/srv/trees/packages/stretch-buildd_vim.tgz stretch debian_buildd-vim

It is essential that you created the debian_buildd-vim directory as a subvolume as outlined above. If not, you won’t be able to create snapshots of it!

This base development install will now serve as the template with which I’ll stamp out the containers. Note that I’m not going to change this install at all, so it’s as vanilla and unopinionated as possible. Any changes will be in the derivative containers of the base template.

systemd-nspawn and machinectl

At this point, we could merely use this installation as a chroot, but that’s not why we’re here. Let’s spin up our container with systemd-nspawn!

root@trout:/srv/trees# systemd-nspawn -D debian_buildd-vim/
Spawning container debian_buildd-vim on /srv/trees/debian_buildd-vim.
Press ^] three times within 1s to kill container.
root@debian_buildd-vim:~#

We’re in like Flynn!

This is the easiest way to spawn a container. There are many switches and options that I am not going to cover here, so read the man page!

That’s terrific, now let’s look at actually booting into a container:

root@trout:/srv/trees# systemd-nspawn -bD debian_buildd-vim/

You should have seen init start and bootstrap the userspace, which then dumps you at a root login:

...
...
...
[  OK  ] Started Console Getty.
[  OK  ] Reached target Login Prompts.
[  OK  ] Started System Logging Service.
[  OK  ] Reached target Multi-User System.
[  OK  ] Reached target Graphical Interface.
     Starting Update UTMP about System Runlevel Changes...
[  OK  ] Started Update UTMP about System Runlevel Changes.

Debian GNU/Linux 9 trout console

trout login:

Ruh roh, we don’t have a login! That’s ok, we’ll just create one. Press Ctrl-]]] to exit the container and then spawn the container without the boot option, -b.

At the prompt, use passwd to set a root password. I suggest 12345. Now, if you spawn the container with the boot option, you can login.

This is great, but remember, this installation is the template, not the container instance we’ll use for changing the world.

Now, let’s copy this dir over to where machinectl expects to find any containers on the system: /var/lib/machines.

cp -r debian_buildd-vim /var/lib/machines

/var/lib/machines should already exist on your system. It is not a regular directory but a mount point of the machines.raw file, which is a btrfs filesystem.

If it doesn’t exist, it is simple enough to create:

  # fallocate -l 20G machines.raw
  # mkfs.btrfs machines.raw
  # mkdir machines
  # mount -o loop machines.raw machines

Commands such as machinectl list-images will list what it finds in this directory (and some others), and this is the location where images will be downloaded when using machinectl pull-tar|pull-raw|import-tar.

Now machinectl knows about this container, which it sees as an image. We can perform operations on any machine listed in this directory, such as clone, start, stop, reboot, etc.

/srv/trees:$ machinectl list-images
NAME              TYPE      RO  USAGE CREATED MODIFIED
debian_buildd-vim directory no  n/a   n/a     n/a

1 images listed.

Note that you can clone a machine using the systemd-nspawn utility, but this won’t place it in /var/lib/machines or make machinectl aware of the image.

For completeness, that command would be:

  btrfs subvolume snapshot debian_buildd-vim nginx

Now, if we want to create a new container from our template, we can issue the following command:

/var/lib:$ machinectl clone debian_buildd-vim nginx
==== AUTHENTICATING FOR org.freedesktop.machine1.manage-images ===
Authentication is required to manage local virtual machine and container images.
Authenticating as: Benjamin Toll,,, (btoll)
Password:
==== AUTHENTICATION COMPLETE ===
/var/lib:$
/var/lib:$ machinectl list-images
NAME              TYPE      RO  USAGE CREATED                     MODIFIED
debian_buildd-vim directory no  n/a   n/a                         n/a
nginx             subvolume no  n/a   Wed 2018-08-22 00:02:17 EDT n/a

2 images listed.

Easy peasy lemon squeezy! You can see that under the hood machinectl created a btrfs subvolume for the nginx machine, since the operating system supports it. Cool.

Now, let’s spawn an instance of the nginx machine and actually install [nginx]:

# systemd-nspawn -M nginx
Spawning container nginx2 on /var/lib/machines/nginx2.
Press ^] three times within 1s to kill container
root@nginx:~# apt-get install nginx
...configure nginx and put content in public dir...

After hitting Ctrl-]]] to kill the container, we’ll export the container as a tar archive which we can upload to the cloud or to a file server on the local network:

# machinectl --format=xz export-tar nginx nginx.xz
Enqueued transfer job 1. Press C-c to continue download in background.
Exporting '/var/lib/machines/nginx', saving to '/var/lib/nginx.xz' with compression 'xz'.
Operation completed successfully.
Exiting.

No big deal.

How do you import a container? Like this:

# machinectl pull-tar https://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-root.tar.gz
# systemd-nspawn -M trusty-server-cloudimg-amd64-root

That one is taken from the machinectl man page. As you can see, pretty standard stuff and what one would expect from a utility to manage containers.

I just ran out of steam. I’ll be updating this as I learn more about systemd-nspawn and machinectl. In the meantime, read the man pages, explore and have fun!

from https://www.benjamintoll.com/2018/08/20/on-systemd-nspawn/

安装chroot

 Easily setup a chroot jail .

Installing chroot

  • Installs Debian

Dependencies

Establish and bootstrap chroot jail

Run install_chroot.sh.

The script will do the following:

  • Install debootstrap and schroot.
  • Create the following chroot definition in /etc/schroot/chroot.d/$CHROOT_NAME:
[$CHROOT_NAME]
description=Debian ($DEBIAN_RELEASE)
type=directory
directory=/srv/chroot/$CHROOT_NAME
users=$CHROOT_USER
groups=sbuild
root-users=$CHROOT_USER
root-groups=root
  • Append the following file(s) to /etc/schroot/default/copyfiles which will then be copied into the jail (note these files must exist in the host environment):
/etc/apt/sources.list
  • Comment-out the /home mount point in /etc/schroot/default/fstab. I don't want /home mounted because:

    • I symlink my dotfiles into my home dir, and they break across filesystems.
    • I want a separate, untethered environment.
  • Create the jail in /srv/chroot/$CHROOT_NAME. It does this by downloading the version of Debian specified on the command line from http://ftp.debian.org/debian.

    Note that if you're bandwidth-impaired like me, you can create the chroot by pointing debootstrap to a mounted image.

    For example:

      mkdir foo
      mount debian_image.iso foo
      debootstrap stretch /srv/chroot/test file:///usr/local/src/iso/debian/free/foo
    

    If you get an error message about GPG, add the --no-check-gpg flag to the debootstrap command.

Make sure to read the schroot(1) and schroot.conf(5) man pages! I've left out a lot of detail here!

For example, I'm configuring the files in /etc/schroot/default/ because I've specified directory as the chroot type in the config in the install_chroot.sh script. If you provide your own chroot config with a different type value, you'll have to alter the scripts in a different dir under /etc/schroot.

That's it, you're done! You can now change (root) to your new chroot by issuing the following commmand:

schroot -u $CHROOT_USER -c $CHROOT_NAME

Note that if proc and dev/pts aren't mounted in the chroot, you will not have a pty when logging in. tmux and other programs will appear not to launch, and when running the tty program, you'll be told not a tty. To fix this, run mnt/chroot_mounts.sh (and its brother mnt/chroot_umounts.sh) in the host environment.

The rest of this document describes optional chroot environment configurations and notes.

Codesharing

  1. Configure chroot

    Run env/codeshare.sh.

    The script will do the following:

    • Install build-essential (gcc, make, etc.), curl, git, tmux and vim.
    • Add the user given on the CLI.
    • Clone my minimal dotfiles into the new home dir:
  2. Configure SSH

    Add the directive to sshd_config that will automatically jail an SSH remote login to the new chroot. Note that you should add this to the ssdh_config in the host environment, NOT the chroot jail.

    Match group codeshare
            ChrootDirectory /srv/chroot/$CHROOT_NAME
            X11Forwarding no
            AllowTcpForwarding no
    

Notes

Installing NodeJS

The NodeJS tarball uses the xz compression tool:

  • sudo apt-get install xz-utils
  • tar xvJf xxx.xz

If getting a "No such file or directory" error when executing the node binary...

The node binary is a 32-bit ELF but it's a 64 bit OS.

The following commands will provide more information:

file /path/to/node
ldd /path/to/node

https://superuser.com/questions/344533/no-such-file-or-directory-error-in-bash-but-the-file-exists

tl;dr:

sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install --reinstall libc6-i386
sudo apt-get install "libstdc++6:i386"

In this example, all users that should be jailed upon remote login should belong to the codeshare group.


from https://github.com/btoll/chroot

-----


https://github.com/skissane/mkjail

利用swoole-socks5翻墙


swoole 实现的socks5代理,可以fq。

swoole 版本大于4.4


from https://github.com/avrilko-go/swoole-socks5

Saturday, 3 April 2021

Linux Atlas


Linux是云计算的主流基础操作系统,除了微软的Azure,其他主要的云计算服务商(Amazon AWS, Goolgle Cloud, Alibaba Cloud …)都构建在各自定制的Linux上。

我个人使用过不同的Linux发行版,虽然各个发行版有各自独特的包管理方式,部分软件的配置惯例有所区别,但总体上技术是相通的。所以在本书「Linux Atlas」撰写中,虽然分了不同的分册面向不同的发行版,但只是因为我在实践过程中恰好使用这个发行版而已。绝大多数的经验技能都可以通用。

从 W3Techs的年度WEB网站Linux统计 Ubuntu使用率最高,这应该和Ubuntu社区活跃安装使用方便有关,其次是CentOS则在大型互联网公司使用较为广泛。总体来说,熟悉这两种主流发行版本对于Linux工作还是非常必要的。知乎上 互联网公司选择 Debian、Ubuntu 和 CentOS 哪一个发行版运维成本最低? 同样引用了W3Techs的报告,可以看出:

  • Ubuntu使用的网站数量高于CentOS,但是更多的网站使用的是Windows以及Unix系统,不过总体来看在Linux范围内Ubuntu的市场份额是CentOS的2倍
  • 大型互联网公司使用CentOS/Ubuntu/Windows的比例差不多
  • 综上:如果是小型互联网公司趋向于使用Ubuntu,大型互联网公司则两者基本持平