Total Pageviews

Saturday, 30 July 2016

Shake is a tool for writing build systems

Shake Hackage version Stackage version Linux Build Status Windows Build Status

Shake is a tool for writing build systems - an alternative to make, Scons, Ant etc. Shake has been used commercially for over five years, running thousands of builds per day. The website for Shake users is at


  • Why use Shake? Shake lets you write large robust build systems, which deal properly with generated source files and run quickly. If you are writing a custom build system of any moderate size (more than a few rules) you should use Shake. The advantages over other build systems are detailed in the document Why choose Shake?.
  • How do I use Shake? Shake is a Haskell library that you use to define your rules. The Shake manual provides a walk through of a small but realistic example, assuming no Haskell knowledge.
  • Generated documentation for all functions, includes lots of examples.
  • Blog posts detailing ongoing development work.
  • Academic paper on the underlying principles behind Shake.

Other links

  • Download the Haskell package from Hackage and install it using Cabal.
  • Mailing list for any questions/bugs/thoughts on Shake. If you need more information and aren't sure where to start, use the mailing list.
  • Questions can be asked on StackOverflow with the tag shake-build-system.
Is your company using Shake? Write something public (even just a tweet to @ndm_haskell) and I'll include a link on the website.


The code under combines fragments from along with documentation from to produce pages. Mostly custom code in the end, with Markdown to parse and convert the pages, followed by TagSoup to tweak them.


"ghc --make" reimplemented with Shake


This is a reimplementation of ghc --make using Shake.  It's entirely one
process, so unlike out-of-process Shake, you won't see any slowdown
compared to --make if you compile with only one core.

# What's the benefit?

    1. You get faster rebuild checking, because Shake doesn't
    recompute the entire dependency graph (ghc --make must
    preprocess and parse all source code every time it loads.)

    2. You can keep going even when one file fails to compile,
    making sure that anything can be built, is built.

    3. You get all the nice Shake features, e.g. unchanging
    rebuilds and profiling.  The profiling is really

# How do I use it?

You'll need these two (somewhat hard to get) items:

    - GHC 8.0

    - A customized version of Shake:

    - Appropriately patched copies of all of Shake's dependencies
      to work with GHC 8.0.  If you wait, these will probably work
      out; but at the time of writing I needed to pass
      --allow-newer=base,transformers,process to cabal to get
      all of the dependencies to install.

One way to to get everything installed (assuming GHC 8.0 is in your
path) is to run:

    git clone
    (cd shake && cabal install --allow-newer=base,transformers,process)
    git clone
    (cd ghc-shake && cabal install --allow-newer=base,transformers,process)

This will enable usage of `ghc --frontend GhcShake`.  We have a sample
script 'ghc-shake' in our source directory which you can pass to Cabal
as a `--with-ghc` argument, which will override the meaning of `--make`
(however, you'll need to edit the `-plugin-package-db` arguments to
accurately reflect your installation.  If you're not building ghc-shake
in place, deleting them should be good enough.)

# What doesn't work

    1. Re-linking detection across packages.  See

    2. Profiling.  See

    3. Linking things that are not executables

    4. Recompilation after package database changes.  (We
    do correctly pick up changes to external interface
    files, however.)

    5. ghc-shake is not really lib-ified properly (it
    should be namespaced and given explicit exports).
    I'll do this eventually.

    6. hs-boot loops don't work; we don't properly
    invalidate the EPS when we successfully compile
    the replacement for an hs file.

If you need something, shout and I'll try to implement it.


Shake is...

  • A build system – an alternative to make, Scons, Ant etc.
  • Reliable and robust – having been relied on commercially for over five years.
  • Powerful – letting you express the problem precisely and directly.
  • Fast to run – both to build from scratch and to rebuild.
Large build systems written using Shake tend to be significantly simpler, while also running faster. If your project can use a canned build system (e.g. Visual Studio, cabal) do that; if your project is very simple use a Makefile; otherwise use Shake.
The original motivation behind the creation of Shake was to allow rules to discover additional dependencies after running previous rules, allowing the build system to generate files and then examine them to determine their dependencies – something that cannot be expressed directly in most build systems. However, now Shake is a suitable build tool even if you do not require that feature.
Click to read about why you should use Shake.

Using Shake

To try Shake you need to install the Haskell Stack, then type stack install shake, then run stack exec -- shake --demo. The final step will create a sample Shake build system and run it (you should see this output).
To write your own Shake build system, read the user manual and refer to the API documentation. Further documentation on specific topics, including more examples, is available from the FAQ. Shake build systems are Haskell programs, but can be treated as a powerful version of make with slightly funny syntax. The build system requires no significant Haskell knowledge, and is designed so that most features are accessible by learning the "Shake syntax", without any appreciation of what the underlying Haskell means.
Click to read the user manual.

Asking questions

Stuck? Confused? Frustrated? Please get in contact! If in doubt, just email the mailing list.
You can ask questions on StackOverflow using the tag shake-build-system. You can email the mailing list with anything about Shake. If you find a bug you can report it at the GitHub issue tracker. If your question needs to remain confidential you can email me, although any insights from your question won't be public, so the other approaches are preferred.

What else?

Shake can execute Ninja files, allowing integration with CMake and Meson. Shake can predict completion time, profile build systems and sanity check builds. Shake is based on a robust underlying theory from this academic paper. Shake is an open source project under the BSD license hosted on GitHub with a range of contributors.

Who uses Shake?

Shake is used by lots of companies, but only a few have declared so publicly.
There are several libraries providing pre-made rules for Shake:
Several open-source projects make key use of Shake:
  • ToolCabal is a rewrite of Cabal using Shake as the dependency engine.
  • ghc-make uses Shake to build programs with GHC, speeding up checking if the build is clean.
  • GHC is in the process of migrating to a Shake-based build system.
  • shake-install helps build a set of cabal packages in the correct order.
And finally, here are a few tutorials and blog posts:

I'm a Haskell programmer who lives in Cambridge with my wife Emily and son Henry (who has grown up a little since the photo on the right was taken). I have a PhD in Computer Science from York University, working on making functional programs shorter, faster and safer. Since then I've worked with F# at Credit Suisse and Haskell/F#/C++ at Standard Chartered, taking the lessons of functional programming and applying them in finance. I'm a strong believer in the functional approach, finding the combination of conciseness, static-typing and testability to offer significant advantages. I've got a blog mostly about Haskell, and I'm also on Twitter, LinkedIn and GitHub. To get in touch email me at

Open Source Projects

I develop a number of open source Haskell projects, all of which can be found at my Github page or on Hackage. I welcome both contributions via pull requests and bug reports via the GitHub issue trackers. Some of my more popular projects include:
  • Shake - a library for writing build systems, an alternative to make.
  • Hoogle - a Haskell API search engine, searching the standard Haskell libraries by function name and type signature.
  • HLint - a tool that suggests stylistic improvements to Haskell code.

Papers and talks

All papers and talks are listed below, most recent first. Show all abstracts or citations.

Talk: Defining your own build system with Shake

How to define your own build system using Shake.

Talk: Shake 'n' Bake

An informal introduction to Shake and Bake.

Talk: Building stuff with Shake

Slides, video, citation, abstract from FP Days 2014, 20 Nov 2014.
A tutorial on Shake, with lots of standalone examples.

Talk: Gluing things together with Haskell

A tour of some of my projects designed to glue code together, replacing shell scripts and Makefiles. Covers Shake, NSIS and Bake.

Talk: Colin's Industrial Influence

Slides, citation, abstract from Runifest, 23 Oct 2014.
A joint talk with Malcolm Wallace covering some projects influenced by Colin Runciman. My slides cover Uniplate and Hoogle.

Paper: Shake Before Building - Replacing Make with Haskell

Paper, slides, video, citation, abstract from ICFP 2012, 10 Sep 2012.
An introduction to Shake, focusing on the theoretical side.

Talk: Hoogle: Finding Functions from Types

Slides, citation from TFP 2011, 16 May 2011.
An overview of how type-search works in Hoogle v1 to v4.

Talk: Shake: A Better Make

Early details about the design of Shake.

Paper: Rethinking Supercompilation

Paper, slides, video, citation, abstract from ICFP 2010, 29 Sep 2010.
Thoughts about making supercompilation faster, based on my experiments with Supero.

Paper: Deriving a Relationship from a Single Example

Assuming you have a single example of a type class, what should the data type be, and what range of instances can be described. Implemented in the Derive tool.

Paper: Losing Functions without Gaining Data

An algorithm for making higher-order programs first-order without introducing new data types (e.g. without doing Reynold's style defunctionalisation). The resulting program may have worse time complexity, but that's fine for certain types of analysis. Implemented as Firstify and used in Catch.

Talk: Supercompilation for Haskell

Slides, citation from Fun in the Afternoon Spring 2009, 03 Mar 2009.
Details of how to apply supercompilation to Haskell.

Paper: Hoogle Overview

Paper, citation, abstract from The Monad.Reader, 19 Nov 2008.
An overview of Hoogle, including both the algorithms and code structure.

Paper: Not All Patterns, But Enough - an automatic verifier for partial but sufficient pattern matching

A static analysis for automatically proving a program will not raise an error at runtime, implemented as Catch. The analysis was able to detect 3 bugs in the HsColour program.

Talk: Hoogle: Fast Type Searching

Slides, audio, citation from AngloHaskell 2008, 09 Aug 2008.
An early overview of how both type and text search works in Hoogle.

Thesis: Transformation and Analysis of Functional Programs

Paper, slides, citation, abstract, 04 Jun 2008.
My PhD thesis, covering a generics library (Uniplate), a supercompiler (Supero), a defunctionalisation algorithm (Firstify) and a pattern-match safety verifier (Catch).

Talk: Instances for Free

Slides, citation from PLASMA, 22 May 2008.
Generating Haskell instances from examples, as found in the Derive tool.

Paper: A Supercompiler for Core Haskell

Paper, citation, abstract from IFL 2007 post proceedings, 01 May 2008.
An early design of the Supero supercompiler. This paper won the Peter Landin prize for the best paper presented at the symposium that year, as selected by the programme committee.

Talk: Detecting Pattern-Match Failures in Haskell

Details about Catch, including worked examples.

Paper: Deriving Generic Functions by Example

An early version of the generic deriving work from Derive.

Paper: Uniform Boilerplate and List Processing

Details of the Uniplate generics library, including information on how to use the Uniplate operations.

Paper: Supero: Making Haskell Faster

Paper, slides, citation, abstract from IFL 2007, 27 Sep 2007.
A very early version of Supero, before it was an actual supercompiler.

Talk: Yhc: Past, Present, Future

Slides, citation from Anglo Haskell 2007, 10 Aug 2007.
The history behind Yhc, along with future plans and intentions.

Talk: Faster Haskell

Slides, citation from Anglo Haskell 2007, 10 Aug 2007.
Early work on Supercompilation.

Talk: Transformation and Analysis of Haskell Source Code

Slides, citation from my thesis seminar, 02 Jul 2007.
Slides giving a very quick overview of my thesis.

Talk: Fastest Lambda First

Slides, citation from PLASMA, 30 May 2007.
Details of a whole-program optimiser for Haskell, early work on Supero.

Paper: Yhc.Core - from Haskell to Core

Paper, citation, abstract from The Monad.Reader, 30 Apr 2007.
Information about the Yhc core language, its constructors and semantics.

Talk: First Order Haskell

Slides, citation from BCTCS 2007, 06 Apr 2007.
An early version of a mechanism for defunctionalising Haskell programs, which later became Firstify.

Talk: Ada: Generics

Slides, citation from the Algorithms and Data Structures course, 07 Mar 2007.
First-year lecture notes on generics in Ada.

Paper: A Static Checker for Safe Pattern Matching in Haskell

Paper, citation, abstract from TFP 2005 post proceedings, 01 Feb 2007.
An early version of Catch.

Talk: Playing with Haskell Data

Slides, citation from PLASMA, 18 Jan 2007.
Generic traversals and transformations over Haskell data types, what would later become Uniplate.

Talk: Haskell With Go Faster Stripes

Slides, citation from PLASMA, 30 Nov 2006.
How to make Haskell faster.

Talk: Hat: Windows and WIMP

Slides, citation from Hat Day 2006 (Kent), 05 Oct 2006.
Thoughts about how the Hat tools could benefit from GUI elements.

Talk: Catch, A Practical Tool

Slides, citation from BCTCS 2006, 06 Apr 2006.
Very early notes on Catch.

Talk: Catch, Lazy Termination

Slides, citation from PLASMA, 16 Mar 2006.
Discussions about what termination means in a lazy language, and how to detect it.

Talk: Hoogle

Slides, citation from PLASMA, 08 Dec 2005.
An early introduction to Hoogle.

Paper: Visual Hat

Paper, slides, citation, abstract from Hat Day 2005 (York), 28 Oct 2005.
Thoughts on how to make the Hat tools work in a GUI.

Paper: Unfailing Haskell: A Static Checker for Pattern Matching

Paper, slides, citation, abstract from TFP 2005, 24 Sep 2005.
A very early version of Catch.

Paper: Qualifying Dissertation: Unfailing Haskell

Paper, citation, abstract, 30 Jun 2005.
Discussions of total functional programming, an an early prototype of Catch.

Talk: Total Pasta

Slides, citation, abstract from BCTCS 2005, 23 Mar 2005.
An algorithm for automatically proving totality of a simple pointer-based programming language.

Talk: Termination checking for a lazy functional language

Slides, citation from my first year literature review seminar, 21 Dec 2004.
A review of the literature around termination checking, particular in lazy languages.

Talk: A New Parser

Slides, citation from PLASMA, 17 Nov 2004.
Thoughts about an alternative way to do parsing.