Source for BiglyBT, a feature filled, open source,
ad-free, bittorrent client. BiglyBT is forked from the original project
and is being maintained by two of the original developers as well as
members of the community. With over 20 years of development, there's a
good chance we have the features you are looking for.
Getting the basics to compile from source is pretty straightforward:
Clone the repo into your favorite IDE
Mark core/src and uis/src as project roots (source modules)
To the uis module, add core/lib/* and one of the swt.jars at /uis/lib/: swt-win64.jar on Windows swt-cocoa-64.jar on OSX swt-linux-64.jar on Linux (GTK)
To the core module, add core/lib/*
Make uis module depend on core. Core should not depend on uis
IntelliJ IDEA will do all these steps for you with its wizard.
External Annotations
If you wish IntelliJ IDEA to show MessageBundle strings
instead of keys, as well as reduce the number of NPE warnings, you can
attach the external annotations either by:
Our binaries and installers up to and including v3.4 are
signed with a digital signature of "Bigly Software". Releases after v3.4
will use an individual signing certificate and will bear the name
"Arron Mogge (Open Source Developer)" denoting the identity of our team
member responsible for signing.
A beginner's guide to setting up a development environment on macOS
macOS Dev Setup
This document describes how I set up my developer
environment on a new MacBook or iMac. We will set up popular programming
languages (for example Node (JavaScript), Python, and Ruby). You may not need all of them for your projects, although I recommend having them set up as they always come in handy.
The document assumes you are new to Mac, but can also be
useful if you are reinstalling a system and need some reminder. The
steps below were tested on macOS High Sierra (10.13), but should work for more recent versions as well.
Contributing: If you find any mistakes in
the steps described below, or if any of the commands are outdated, do
let me know! For any other suggestions, please understand if I don't
include everything. This guide was originally written for some friends
getting started with programming on a Mac, and as a personal reference
for myself. I'm trying to keep it simple!
First thing you need to do, on any OS actually, is update the system! For that: Apple Icon > About This Mac then Software Update....
System preferences
If this is a new computer, there are a couple of tweaks I
like to make to the System Preferences. Feel free to follow these, or to
ignore them, depending on your personal preferences.
In Apple Icon > System Preferences:
Trackpad > Tap to click
Keyboard > Key Repeat > Fast (all the way to the right)
Keyboard > Delay Until Repeat > Short (all the way to the right)
Dock > Automatically hide and show the Dock
Security
I recommend checking that basic security settings are
enabled. You will be happy to have done so if ever your Mac is lost or
stolen.
In Apple Icon > System Preferences:
Users & Groups: If you haven't already set a password for your user during the initial set up, you should do so now
Security & Privacy > General: Require password immediately
after sleep or screen saver begins (you can keep a grace period of a
couple minutes if you prefer, but I like to know that my computer locks
as soon as I close it)
Security & Privacy > FileVault: Make sure FileVault disk encryption is enabled
iCloud: If you haven't already done so during set up, enable Find My Mac
iTerm2
Install
Since we're going to be spending a lot of time in the
command-line, let's install a better terminal than the default one.
Download and install iTerm2.
In Finder, drag and drop the iTerm Application file into the Applications folder.
You can now launch iTerm, through the Launchpad for instance.
Let's just quickly change some preferences. In iTerm2 > Preferences..., under the tab General, uncheck Confirm closing multiple sessions and Confirm "Quit iTerm2 (Cmd+Q)" command under the section Closing.
In the tab Profiles, create a new one with the "+" icon, and rename it to your first name for example. Then, select Other Actions... > Set as Default. Under the section General set Working Directory to be Reuse previous session's directory. Finally, under the section Window, change the size to something better, like Columns: 125 and Rows: 35.
When done, hit the red "X" in the upper left (saving is
automatic in macOS preference panes). Close the window and open a new
one to see the size change.
Beautiful terminal
Since we spend so much time in the terminal, we should try
to make it a more pleasant and colorful place. What follows might seem
like a lot of work, but trust me, it'll make the development experience
so much better.
First let's add some color. There are many great color schemes out there, but if you don't know where to start you can try Atom One Dark. Download the iTerm presets for the theme by running:
cd ~/Downloads
curl -o "Atom One Dark.itermcolors" https://raw.githubusercontent.com/nathanbuchar/atom-one-dark-terminal/master/scheme/iterm/One%20Dark.itermcolors
curl -o "Atom One Light.itermcolors" https://raw.githubusercontent.com/nathanbuchar/atom-one-dark-terminal/master/scheme/iterm/One%20Light.itermcolors
Then, in iTerm2 Preferences, under Profiles and Colors, go to Color Presets... > Import..., find and open the Atom One Dark.itermcolors file we just downloaded. Repeat these steps for Atom One Light.itermcolors. Now open Color Presets... again and select Atom One Dark to activate the dark theme (or choose the light them if that's your preference).
Not a lot of colors yet. We need to tweak a little bit our
Unix user's profile for that. This is done (on macOS and Linux), in the
~/.bash_profile text file (~ stands for the user's home directory).
We'll come back to the details of that later, but for now, just download the files .bash_profile, .bash_prompt, .aliases attached to this document into your home directory (.bash_profile is the one that gets loaded, I've set it up to call the others):
With that, open a new terminal tab (Cmd+T) and see the change! Try the list commands: ls, ls -lh (aliased to ll), ls -lha (aliased to la).
Now we have a terminal we can work with!
(Thanks to Mathias Bynens for his awesome dotfiles.)
Homebrew
Package managers make it so much easier to install and
update applications (for Operating Systems) or libraries (for
programming languages). The most popular one for macOS is Homebrew.
Install
An important dependency before Homebrew can work is the Command Line Developer Tools for Xcode. These include compilers that will allow you to build things from source. You can install them directly from the terminal with:
xcode-select --install
Once that is done, we can install Homebrew by copy-pasting the installation command from the Homebrew homepage inside the terminal:
Follow the steps on the screen. You will be prompted for your user password so Homebrew can set up the appropriate permissions.
Once installation is complete, you can run the following command to make sure everything works:
brew doctor
Usage
To install a package (or Formula in Homebrew vocabulary) simply type:
brew install <formula>
To see if any of your packages need to be updated:
brew outdated
To update a package:
brew upgrade <formula>
Homebrew keeps older versions of packages installed, in
case you want to rollback. That rarely is necessary, so you can do some
cleanup to get rid of those old versions:
brew cleanup
To see what you have installed (with their version numbers):
brew list --versions
Homebrew Services
A nice extension to Homebrew is Homebrew Services. It will automatically launch things like databases when your computer starts, so you don't have to do it manually every time.
Homebrew Services will automatically install itself the first time you run it, so there is nothing special to do.
After installing a service (for example a database), it
should automatically add itself to Homebrew Services. If not, you can
add it manually with:
brew services <formula>
Start a service with:
brew services start <formula>
At anytime you can view which services are running with:
brew services list
Git
macOS comes with a pre-installed version of Git,
but we'll install our own through Homebrew to allow easy upgrades and
not interfere with the system version. To do so, simply run:
brew install git
When done, to test that it installed fine you can run:
which git
The output should be /usr/local/bin/git.
Let's set up some basic configuration. Download the .gitconfig file to your home directory:
cd ~
curl -O https://raw.githubusercontent.com/nicolashery/mac-dev-setup/master/.gitconfig
It will add some color to the status, branch, and diff Git commands, as well as a couple aliases. Feel free to take a look at the contents of the file, and add to it to your liking.
Next, we'll define your Git user (should be the same name and email you use for GitHub and Heroku):
On a Mac, it is important to remember to add .DS_Store (a hidden macOS system file that's put in folders) to your project .gitignore files. You also set up a global .gitignore file, located for instance in your home directory (but you'll want to make sure any collaborators also do it):
cd ~
curl -O https://raw.githubusercontent.com/nicolashery/mac-dev-setup/master/.gitignore
git config --global core.excludesfile ~/.gitignore
Visual Studio Code
With the terminal, the text editor is a developer's most
important tool. Everyone has their preferences, but if you're just
getting started and looking for something simple that works, Visual Studio Code is a pretty good option.
Go ahead and download it. Open the .dmg file, drag-and-drop in the Applications folder, you know the drill now. Launch the application.
Note: At this point I'm going to create a
shortcut on the macOS Dock for both for Visual Studio Code and iTerm.
To do so, right-click on the running application and select Options > Keep in Dock.
Just like the terminal, let's configure our editor a little. Go to Code > Preferences > Settings. In the very top-right of the interface you should see an icon with brackets that appeared { } (on hover, it should say "Open Settings (JSON)"). Click on it, and paste the following:
Feel free to tweak these to your preference. When done, save the file and close it.
Pasting the above JSON snippet was handy to quickly
customize things, but for further setting changes feel free to search in
the "Settings" panel that opened first (shortcut Cmd+,). When you're happy with your setup, you can save the JSON to quickly restore it on a new machine.
If you remember only one keyboard shortcut in VS Code, it should be Cmd+Shift+P. This opens the Command Palette, from which you can run pretty much anything.
Let's open the command palette now, and search for Shell Command: Install 'code' command in PATH. Hit enter when it shows up. This will install the command-line tool code to quickly open VS Code from the terminal. When in a projects directory, you'll be able to run:
cd myproject/
code .
VS Code is very extensible. To customize it further, open the Extensions tab on the left.
Let's do that now to customize the color of our editor. Search for the Atom One Dark Theme extension, select it and click Install. Repeat this for the Atom One Light Theme.
Finally, activate the theme by going to Code > Preferences > Color Theme and selecting Atom One Dark (or Atom One Light if that is your preference).
Vim
Although VS Code will be our main editor, it is a good idea to learn some very basic usage of Vim. It is a very popular text editor inside the terminal, and is usually pre-installed on any Unix system.
For example, when you run a Git commit, it will open Vim to allow you to type the commit message.
I suggest you read a tutorial on Vim. Grasping the concept of the two "modes" of the editor, Insert (by pressing i) and Normal (by pressing Esc to exit Insert mode), will be the part that feels most unnatural. Also, it is good to know that typing :x when in Normal mode will save and exit. After that, it's just remembering a few important keys.
Vim's default settings aren't great, and you could spend a lot of time tweaking your configuration (the .vimrc file). But if you only use Vim occasionally, you'll be happy to know that Tim Pope has put together some sensible defaults to quickly get started.
Using Vim's built-in package support, install these settings by running:
mkdir -p ~/.vim/pack/tpope/start
cd ~/.vim/pack/tpope/start
git clone https://tpope.io/vim/sensible.git
With that, Vim will look a lot better next time you open it!
Python
macOS, like Linux, ships with Python
already installed. But you don't want to mess with the system Python
(some system tools rely on it, etc.), so we'll install our own version
using pyenv. This will also allow us to manage multiple versions of Python (ex: 2.7 and 3) should we need to.
Install pyenv via Homebrew by running:
brew install pyenv
When finished, you should see instructions to add something to your profile. Open your .bash_profile in the home directory (you can use code ~/.bash_profile), and add the following line:
Before installing a new Python version, the pyenv wiki recommends having a few dependencies available:
brew install openssl readline sqlite3 xz zlib
We can now list all available Python versions by running:
pyenv install --list
Look for the latest 3.x version (or 2.7.x), and install it (replace the .x.x with actual numbers):
pyenv install 3.x.x
List the Python versions you have locally with:
pyenv versions
The star (*) should indicate we are still using the system version, which is the default. I recommend leaving it as the default as some Node.js packages will use it in their installation process.
You can switch your current terminal to another Python version with:
pyenv shell 3.x.x
You should now see that version when running:
python --version
In a project directory, you can use:
pyenv local 3.x.x
This will save that project's Python version to a .python-version file. Next time you enter the project's directory from a terminal, pyenv will automatically load that version for you.
For more information, see the pyenv commands documentation.
pip
pip was also installed by pyenv. It is the package manager for Python.
Here are a couple Pip commands to get you started. To install a Python package:
pip install <package>
To upgrade a package:
pip install --upgrade <package>
To see what's installed:
pip freeze
To uninstall a package:
pip uninstall <package>
virtualenv
virtualenv is a tool that creates an isolated Python environment for each of your projects.
For a particular project, instead of installing required
packages globally, it is best to install them in an isolated folder,
that will be managed by virtualenv. The advantage is that
different projects might require different versions of packages, and it
would be hard to manage that if you install packages globally.
Instead of installing and using virtualenv directly, we'll use the dedicated pyenv plugin pyenv-virtualenv which will make things a bit easier for us. Install it via Homebrew:
brew install pyenv-virtualenv
After installation, add the following line to your .bash_profile:
if which pyenv-virtualenv-init > /dev/null;theneval"$(pyenv virtualenv-init -)";fi
And reload it with:
source ~/.bash_profile
Now, let's say you have a project called myproject. You can set up a virtualenv for that project and the Python version it uses (replace 3.x.x with the version you want):
pyenv virtualenv 3.x.x myproject
See the list of virtualenvs you created with:
pyenv virtualenvs
To use your project's virtualenv, you need to activate it first (in every terminal where you are working on your project):
pyenv activate myproject
If you run pyenv virtualenvs again, you should see a star (*) next to the active virtualenv.
Now when you install something:
pip install <package>
It will get installed in that virtualenv's folder, and not conflict with other projects.
You can also set your project's .python-version to point to a virtualenv you created:
pyenv local myproject
Next time you enter that project's directory, pyenv will automatically load the virtualenv for you.
Anaconda and Miniconda
The Anaconda/Miniconda distributions of Python come with many useful tools for scientific computing.
You can install them using pyenv, for example (replace x.x.x with an actual version number):
pyenv install miniconda3-x.x.x
After loading an Anaconda or Miniconda Python distribution into your shell, you can create conda environments (which are similar to virtualenvs):
The recommended way to install Node.js is to use nvm (Node Version Manager) which allows you to manage multiple versions of Node.js on the same machine.
To install a package and save it in your project's package.json file:
npm install --save <package>
To see what's installed:
npm list --depth 1 # Local packages
npm list -g --depth 1 # Global packages
To find outdated packages (locally or globally):
npm outdated [-g]
To upgrade all or a particular package:
npm update [<package>]
To uninstall a package:
npm uninstall --save <package>
Ruby
Like Python, Ruby
is already installed on Unix systems. But we don't want to mess around
with that installation. More importantly, we want to be able to use the
latest version of Ruby.
Install
The recommended way to install Ruby is to use rbenv, which allows you to manage multiple versions of Ruby on the same machine. You can install rbenv with Homebrew:
brew install rbenv
After installation, add the following line to your .bash_profile:
eval"$(rbenv init -)"
And reload it with:
source ~/.bash_profile
Usage
The following command will show you which versions of Ruby are available to install:
rbenv install --list
You can find the latest version in that list and install it with (replace .x.x with actual version numbers):
rbenv install 2.x.x
Run the following to see which versions you have installed:
rbenv versions
The start (*) will show you that we are currently using the default system version. You can switch your terminal to use the one you just installed:
rbenv shell 2.x.x
You can also set it as the default version if you want:
rbenv global 2.x.x
In a specific project's directory, you can ask rbenv to create a .ruby-version file. Next time you enter that project's directory from the terminal, it will automatically load the correct Ruby version:
RubyGems, the Ruby package manager, was also installed:
which gem
The first thing you want to do after installing a new Ruby version is to install Bundler.
This tool will allow you to set up separate environments for your
different Ruby projects, so their required gem versions won't conflict
with each other. Install Bundler with:
gem install bundler
In a new Ruby project directory, create a new Gemfile with:
bundle init
Add a dependency to the Gemfile, for example the Jekyll static site generator:
source"https://rubygems.org"gem"jekyll"
Then install the project's dependencies with:
bundle install
Make sure you check in both the Gemfile and Gemfile.lock into your Git repository.
Heroku is a Platform-as-a-Service
(PaaS) that makes it really easy to deploy your apps. There are other
similar solutions out there, but Heroku is among the most popular. Not
only does it make a developer's life easier, but I find that having
Heroku deployment in mind when building an app forces you to follow
modern app development best practices.
Assuming that you have an account (sign up if you don't), let's install the Heroku CLI:
brew tap heroku/brew
brew install heroku
Login to your Heroku account using:
heroku login
(This will prompt you to open a page in your web browser and log in to your Heroku account.)
Once logged-in, you're ready to deploy apps! Heroku has great Getting Started
guides for different languages, so I'll let you refer to that. Heroku
uses Git to push code for deployment, so make sure your app is under Git
version control. A quick cheat sheet (if you've used Heroku before):
The Heroku Dev Center is full of great resources, so be sure to check it out!
PostgreSQL
PostgreSQL is a popular relational database, and Heroku has first-class support for it.
Install PostgreSQL using Homebrew:
brew install postgresql
It will automatically add itself to Homebrew Services. Start it with:
brew services start postgresql
If you reboot your machine, PostgreSQL will be restarted at login.
GUI
You can interact with your SQL database by running psql in the terminal.
If you prefer a GUI (Graphical User Interface), Postico has a simple free version that let's you explore tables and run SQL queries.
Redis
Redis is a
fast, in-memory, key-value store, that uses the disk for persistence. It
complements nicely a database such as PostgreSQL. There are a lot of interesting things
that you can do with it. For example, it's often used for session
management or caching by web apps, but it has many other uses.
To install Redis, use Homebrew:
brew install redis
Start it through Homebrew Services with:
brew services start redis
I'll let you refer to Redis' documentation or other tutorials for more information.
Elasticsearch
Elasticsearch
is a distributed search and analytics engine. It uses an HTTP REST API,
making it easy to work with from any programming language.
You can use elasticsearch for things such as real-time
search results, autocomplete, recommendations, machine learning, and
more.
Install
Elasticsearch runs on Java, so check if you have it installed by running:
java -version
If Java isn't installed yet, dismiss the window that just appeared by clicking "Ok", and install Java via Homebrew:
brew cask install homebrew/cask-versions/java8
Next, install Elasticsearch with:
brew install elasticsearch
Usage
Start the Elasticsearch server with:
brew services start elasticsearch
Test that the server is working correctly by running:
curl -XGET 'http://localhost:9200/'
(You may need to wait a little bit for it to boot up if you just started the service.)
You can interact with the Elasticsearch server using curl, or anything that can send an HTTP request.
However, if you prefer a graphical interface, you can take a look at Dejavu. You can easily install it via the Dejavu Chrome Extension.
Projects folder
This really depends on how you want to organize your files, but I like to put all my version-controlled projects in ~/Projects. Other documents I may have, or things not yet under version control, I like to put in ~/Dropbox (if you have Dropbox installed), or ~/Documents if you prefer to use iCloud Drive.
Apps
Here is a quick list of some apps I use, and that you might find useful as well:
1Password: Securely store your login and passwords, and access them from all your devices. ($3/month)
Dropbox: File syncing to the cloud. It is cross-platform, but if all your devices are Apple you may prefer iCloud Drive. (Free for 2GB)
Postman: Easily make HTTP requests. Useful to test your REST APIs. (Free for basic features)
GitHub Desktop: I do everything through the git command-line tool, but I like to use GitHub Desktop just to review the diff of my changes. (Free)
Spectacle: Move and resize windows with keyboard shortcuts. (Free)
Fullmoon项目是一款基于苹果生态的 iOS/macOS 本地大语言模型（LLM）聊天应用。它利用 Apple Silicon 强大的算力和 Apple 开源的 MLX Swift 框架，实现了在设备端离线运行 LLM，保证了数据隐私的安全性。遵循 MIT 开源协议.项目不仅提供了 iOS 客户端，还包含了 macOS 和 visionOS 的适配代码。
Fullmoon 核心定位是 “私有 LLM 的最简使用方式”。与传统的云端 AI 不同，Fullmoon 允许用户在 iPhone、iPad、Mac 甚至 Vision Pro 等 Apple 设备上，完全离线 地运行大语言模型。这意味着用户的对话内容不会发送到服务器，而是安全地保存在本地，摆脱了对网络连接的依赖。
Fullmoon 的实现得益于苹果对机器学习框架的深度优化。它基于 Apple 官方开源的 MLX Swift 框架。
MLX Swift：这是 Apple 在 WWDC 2025 推出的机器学习框架，专为 Apple Silicon 设计。它允许开发者在 Swift 环境中直接调用机器学习模型，利用 Metal 3 图形框架进行高效计算。 硬件加速：Fullmoon 通过底层调用 Metal，直接在 iPhone、iPad 或 Mac 的 GPU 上运行模型，提供了比传统 CPU 推理更快的速度，同时大幅降低了能耗。
XNU kernel is part of the Darwin operating system for use
in macOS and iOS operating systems. XNU is an acronym for X is Not Unix.
XNU is a hybrid kernel combining the Mach kernel developed at Carnegie
Mellon University with components from FreeBSD and a C++ API for writing
drivers called IOKit.
XNU runs on x86_64 for both single processor and multi-processor
configurations.
XNU Source Tree
config - configurations for exported apis for supported architecture and platform
SETUP - Basic set of tools used for configuring the kernel, versioning and kextsymbol management.
EXTERNAL_HEADERS - Headers sourced from other projects
to avoid dependency cycles when building. These headers should be
regularly synced when source is updated.
libkern - C++ IOKit library code for handling of drivers and kexts.
libsa - kernel bootstrap code for startup
libsyscall - syscall library interface for userspace programs
libkdd - source for user library for parsing kernel data like kernel chunked data.
makedefs - top level rules and defines for kernel build.
osfmk - Mach kernel based subsystems
pexpert - Platform specific code like interrupt handling, atomics etc.
security - Mandatory Access Check policy interfaces and related implementation.
bsd - BSD subsystems code
tools - A set of utilities for testing, debugging and profiling kernel.
How to build XNU
Building DEVELOPMENT kernel
The xnu make system can build kernel based on KERNEL_CONFIGS & ARCH_CONFIGS variables as arguments.
Here is the syntax:
make SDKROOT=<sdkroot> ARCH_CONFIGS=<arch> KERNEL_CONFIGS=<variant>
Where:
<sdkroot>: path to macOS SDK on disk. (defaults to /)
<variant>: can be debug, development, release, profile and configures compilation flags and asserts throughout kernel code.
<arch> : can be valid arch to build for. (E.g. X86_64)
To build a kernel for the same architecture as running OS, just type
$ make
$ make SDKROOT=macosx.internal
Additionally, there is support for configuring architectures through ARCH_CONFIGS and kernel configurations with KERNEL_CONFIGS.
$ make SDKROOT=macosx.internal ARCH_CONFIGS=X86_64 KERNEL_CONFIGS=DEVELOPMENT
$ make SDKROOT=macosx.internal ARCH_CONFIGS=X86_64 KERNEL_CONFIGS="RELEASE DEVELOPMENT DEBUG"
Note:
By default, architecture is set to the build machine architecture, and the default kernel
config is set to build for DEVELOPMENT.
This will also create a bootable image, kernel.[config], and a kernel binary
with symbols, kernel.[config].unstripped.
To intall the kernel into a DSTROOT, use the install_kernels target:
$ make install_kernels DSTROOT=/tmp/xnu-dst
Hint:
For a more satisfying kernel debugging experience, with access to all
local variables and arguments, but without all the extra check of the
DEBUG kernel, add something like:
CFLAGS_DEVELOPMENTARM64="-O0 -g -DKERNEL_STACK_MULTIPLIER=2"
CXXFLAGS_DEVELOPMENTARM64="-O0 -g -DKERNEL_STACK_MULTIPLIER=2"
to your make command.
Replace DEVELOPMENT and ARM64 with the appropriate build and platform.
To build with RELEASE kernel configuration
make KERNEL_CONFIGS=RELEASE SDKROOT=/path/to/SDK
Building FAT kernel binary
Define architectures in your environment or when running a make command.
$ make ARCH_CONFIGS="X86_64" exporthdrs all
Other makefile options
$ make MAKEJOBS=-j8 # this will use 8 processes during the build. The default is 2x the number of active CPUS.
$ make -j8 # the standard command-line option is also accepted
$ make -w # trace recursive make invocations. Useful in combination with VERBOSE=YES
$ make BUILD_LTO=0 # build without LLVM Link Time Optimization
$ make REMOTEBUILD=user@remotehost # perform build on remote host
$ make BUILD_JSON_COMPILATION_DATABASE=1 # Build Clang JSON Compilation Database
The XNU build system can optionally output color-formatted build output. To enable this, you can either
set the XNU_LOGCOLORS environment variable to y, or you can pass LOGCOLORS=y to the make command.
Debug information formats
By default, a DWARF debug information repository is
created during the install phase; this is a "bundle" named
kernel.development.<variant>.dSYM
To select the older STABS debug information format (where debug
information is embedded in the kernel.development.unstripped image), set
the BUILD_STABS environment variable.
$ export BUILD_STABS=1
$ make
Building KernelCaches
To test the xnu kernel, you need to build a kernelcache that links the kexts and
kernel together into a single bootable image.
To build a kernelcache you can use the following mechanisms:
Using automatic kernelcache generation with kextd.
The kextd daemon keeps watching for changing in /System/Library/Extensions directory.
So you can setup new kernel as
The development kernel and iBoot supports configuring boot
arguments so that we can safely boot into test kernel and, if things go
wrong, safely fall back to previously used kernelcache.
Following are the steps to get such a setup:
Create kernel cache using the kextcache command as /kernelcache.test
Copy exiting boot configurations to alternate file
The --nextonly flag specifies that use the boot.plist configs only for one boot.
So if the kernel panic's you can easily power reboot and recover back to original kernel.
Creating tags and cscope
Set up your build environment and from the top directory, run:
$ make tags # this will build ctags and etags on a case-sensitive volume, only ctags on case-insensitive
$ make TAGS # this will build etags
$ make cscope # this will build cscope database
XNU installs header files at the following locations -
a. $(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers
b. $(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders
c. $(DSTROOT)/usr/include/
d. $(DSTROOT)/System/DriverKit/usr/include/
e. $(DSTROOT)/System/Library/Frameworks/System.framework/PrivateHeaders
Kernel.framework is used by kernel extensions.
The System.framework and /usr/include are used by user level applications. /System/DriverKit/usr/include is used by userspace drivers.
The header files in framework's PrivateHeaders are only available for ** Apple Internal Development **.
The directory containing the header file should have a Makefile that
creates the list of files that should be installed at different locations.
If you are adding the first header file in a directory, you will need to
create Makefile similar to xnu/bsd/sys/Makefile.
Add your header file to the correct file list depending on where you want
to install it. The default locations where the header files are installed
from each file list are -
a. `DATAFILES` : To make header file available in user level -
`$(DSTROOT)/usr/include`
b. `DRIVERKIT_DATAFILES` : To make header file available to DriverKit userspace drivers -
`$(DSTROOT)/System/DriverKit/usr/include`
c. `PRIVATE_DATAFILES` : To make header file available to Apple internal in
user level -
`$(DSTROOT)/System/Library/Frameworks/System.framework/PrivateHeaders`
d. `KERNELFILES` : To make header file available in kernel level -
`$(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers`
`$(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders`
e. `PRIVATE_KERNELFILES` : To make header file available to Apple internal
for kernel extensions -
`$(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders`
The Makefile combines the file lists mentioned above into different
install lists which are used by build system to install the header files. There
are two types of install lists: machine-dependent and machine-independent.
These lists are indicated by the presence of MD and MI in the build
setting, respectively. If your header is architecture-specific, then you should
use a machine-dependent install list (e.g. INSTALL_MD_LIST). If your header
should be installed for all architectures, then you should use a
machine-independent install list (e.g. INSTALL_MI_LIST).
If the install list that you are interested does not exist, create it
by adding the appropriate file lists. The default install lists, its
member file lists and their default location are described below -
a. `INSTALL_MI_LIST` : Installs header file to a location that is available to everyone in user level.
Locations -
$(DSTROOT)/usr/include
Definition -
INSTALL_MI_LIST = ${DATAFILES}
b. `INSTALL_DRIVERKIT_MI_LIST` : Installs header file to a location that is
available to DriverKit userspace drivers.
Locations -
$(DSTROOT)/System/DriverKit/usr/include
Definition -
INSTALL_DRIVERKIT_MI_LIST = ${DRIVERKIT_DATAFILES}
c. `INSTALL_MI_LCL_LIST` : Installs header file to a location that is available
for Apple internal in user level.
Locations -
$(DSTROOT)/System/Library/Frameworks/System.framework/PrivateHeaders
Definition -
INSTALL_MI_LCL_LIST = ${PRIVATE_DATAFILES}
d. `INSTALL_KF_MI_LIST` : Installs header file to location that is available
to everyone for kernel extensions.
Locations -
$(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers
Definition -
INSTALL_KF_MI_LIST = ${KERNELFILES}
e. `INSTALL_KF_MI_LCL_LIST` : Installs header file to location that is
available for Apple internal for kernel extensions.
Locations -
$(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders
Definition -
INSTALL_KF_MI_LCL_LIST = ${KERNELFILES} ${PRIVATE_KERNELFILES}
f. `EXPORT_MI_LIST` : Exports header file to all of xnu (bsd/, osfmk/, etc.)
for compilation only. Does not install anything into the SDK.
Definition -
EXPORT_MI_LIST = ${KERNELFILES} ${PRIVATE_KERNELFILES}
g. `INSTALL_MODULEMAP_INCDIR_MI_LIST` : Installs module map file to a
location that is available to everyone in user level, installing at the
root of INCDIR.
Locations -
$(DSTROOT)/usr/include
Definition -
INSTALL_MODULEMAP_INCDIR_MI_LIST = ${MODULEMAP_INCDIR_FILES}
If you want to install the header file in a sub-directory of the paths
described in (1), specify the directory name using two variables
INSTALL_MI_DIR and EXPORT_MI_DIR as follows -
INSTALL_MI_DIR = dirname
EXPORT_MI_DIR = dirname
A single header file can exist at different locations using the steps
mentioned above. However it might not be desirable to make all the code
in the header file available at all the locations. For example, you
want to export a function only to kernel level but not user level.
You can use C language's pre-processor directive (#ifdef, #endif, #ifndef)
to control the text generated before a header file is installed. The kernel
only includes the code if the conditional macro is TRUE and strips out
code for FALSE conditions from the header file.
Some pre-defined macros and their descriptions are -
a. `PRIVATE` : If defined, enclosed definitions are considered System
Private Interfaces. These are visible within xnu and
exposed in user/kernel headers installed within the AppleInternal
"PrivateHeaders" sections of the System and Kernel frameworks.
b. `KERNEL_PRIVATE` : If defined, enclosed code is available to all of xnu
kernel and Apple internal kernel extensions and omitted from user
headers.
c. `BSD_KERNEL_PRIVATE` : If defined, enclosed code is visible exclusively
within the xnu/bsd module.
d. `MACH_KERNEL_PRIVATE`: If defined, enclosed code is visible exclusively
within the xnu/osfmk module.
e. `XNU_KERNEL_PRIVATE`: If defined, enclosed code is visible exclusively
within xnu.
f. `KERNEL` : If defined, enclosed code is available within xnu and kernel
extensions and is not visible in user level header files. Only the
header files installed in following paths will have the code -
$(DSTROOT)/System/Library/Frameworks/Kernel.framework/Headers
$(DSTROOT)/System/Library/Frameworks/Kernel.framework/PrivateHeaders
g. `DRIVERKIT`: If defined, enclosed code is visible exclusively in the
DriverKit SDK headers used by userspace drivers.
Conditional compilation
xnu offers the following mechanisms for conditionally compiling code:
a. *CPU Characteristics* If the code you are guarding has specific
characterstics that will vary only based on the CPU architecture being
targeted, use this option. Prefer checking for features of the
architecture (e.g. `__LP64__`, `__LITTLE_ENDIAN__`, etc.).
b. *New Features* If the code you are guarding, when taken together,
implements a feature, you should define a new feature in `config/MASTER`
and use the resulting `CONFIG` preprocessor token (e.g. for a feature
named `config_virtual_memory`, check for `#if CONFIG_VIRTUAL_MEMORY`).
This practice ensures that existing features may be brought to other
platforms by simply changing a feature switch.
c. *Existing Features* You can use existing features if your code is
strongly tied to them (e.g. use `SECURE_KERNEL` if your code implements
new functionality that is exclusively relevant to the trusted kernel and
updates the definition/understanding of what being a trusted kernel means).
It is recommended that you avoid compiling based on the target platform. xnu
does not define the platform macros from TargetConditionals.h
(TARGET_OS_OSX, TARGET_OS_IOS, etc.).
There is a deprecated TARGET_OS_EMBEDDED macro, but this should be avoided
as it is in general too broad a definition for most functionality.
Please refer to TargetConditionals.h for a full picture.
How to add a new syscall
Testing the kernel
XNU kernel has multiple mechanisms for testing.
Assertions - The DEVELOPMENT and DEBUG kernel configs are compiled with assertions enabled. This allows developers to easily
test invariants and conditions.
XNU Power On Self Tests (XNUPOST): The XNUPOST config allows for building the kernel with basic set of test functions
that are run before first user space process is launched. Since XNU is hybrid between MACH and BSD, we have two locations where
tests can be added.
xnu/osfmk/tests/ # For testing mach based kernel structures and apis.
bsd/tests/ # For testing BSD interfaces.
User level tests: The tools/tests/ directory holds all the tests that verify syscalls and other features of the xnu kernel.
The make target xnu_tests can be used to build all the tests supported.
$ make RC_ProjectName=xnu_tests SDKROOT=/path/to/SDK
These tests are individual programs that can be run from
Terminal and report tests status by means of std posix exit codes (0
-> success) and/or stdout.
Please read detailed documentation in tools/tests/unit_tests/README.md
Kernel data descriptors
XNU uses different data formats for passing data in its
api. The most standard way is using syscall arguments. But for complex
data
it often relies of sending memory saved by C structs. This packaged data
transport mechanism is fragile and leads to broken interfaces
between user space programs and kernel apis. libkdd directory holds user space library that can parse custom data provided by the
same version of kernel. The kernel chunked data format is described in detail at libkdd/README.md.
Debugging the kernel
The xnu kernel supports debugging with a remote kernel debugging protocol (kdp). Please refer documentation at [technical note] TN2063
By default the kernel is setup to reboot on a panic. To debug a live
kernel, the kdp server is setup to listen for UDP connections
over ethernet. For machines without ethernet port, this behavior can be
altered with use of kernel boot-args. Following are some
common options.
debug=0x144 - setups debug variables to start kdp debugserver on panic
-v - print kernel logs on screen. By default XNU only shows grey screen with boot art.
kdp_match_name=en1 - Override default port selection for kdp. Supported for ethernet, thunderbolt and serial debugging.
To debug a panic'ed kernel, use llvm debugger (lldb) along with unstripped symbol rich kernel binary.
sh$ lldb kernel.development.unstripped
And then you can connect to panic'ed machine with kdp_remote [ip addr] or gdb_remote [hostip : port] commands.
Each kernel is packaged with kernel specific debug scripts
as part of the build process. For security reasons these special
commands
and scripts do not get loaded automatically when lldb is connected to
machine. Please add the following setting to your ~/.lldbinit
if you wish to always load these macros.
settings set target.load-script-from-symbol-file true
The tools/lldbmacros directory contains the source for each of these commands. Please follow the README.md
for detailed explanation of commands and their usage.
