wav2letter

Facebook AI Research's Automatic Speech Recognition Toolkit.

https://github.com/facebookresearch/wav2letter/wiki

 

Important Note:

wav2letter has been moved and consolidated into Flashlight in the ASR application.

Future wav2letter development will occur in Flashlight.

To build the old, pre-consolidation version of wav2letter, checkout the wav2letter v0.2 release, which depends on the old Flashlight v0.2 release. The wav2letter-lua project can be found on the wav2letter-lua branch, accordingly.

For more information on wav2letter++, see or cite this arXiv paper.

Recipes

This repository includes recipes to reproduce the following research papers as well as pre-trained models. All results reproduction must use Flashlight <= 0.3.2 for exact reproducability. Papers contained here include:

• Pratap et al. (2020): Scaling Online Speech Recognition Using ConvNets
• Synnaeve et al. (2020): End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures
• Kahn et al. (2020): Self-Training for End-to-End Speech Recognition
• Likhomanenko et al. (2019): Who Needs Words? Lexicon-free Speech Recognition
• Hannun et al. (2019): Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions

Data preparation for training and evaluation can be found in data directory.

Building the Recipes

First, install Flashlight (using the 0.3 branch is required) with the ASR application.

mkdir build && cd build
cmake .. && make -j8

If Flashlight or ArrayFire are installed in nonstandard paths via a custom CMAKE_INSTALL_PREFIX, they can be found by passing

-Dflashlight_DIR=[PREFIX]/usr/share/flashlight/cmake/ -DArrayFire_DIR=[PREFIX]/usr/share/ArrayFire/cmake

when running cmake.

Join the wav2letter community

• Facebook page: https://www.facebook.com/groups/717232008481207/
• Google group: https://groups.google.com/forum/#!forum/wav2letter-users
• Contact: vineelkpratap@fb.com, awni@fb.com, qiantong@fb.com, jacobkahn@fb.com, antares@fb.com, avidov@fb.com, gab@fb.com, vitaliy888@fb.com, locronan@fb.com

from https://github.com/flashlight/wav2letter

------

Important Note:

wav2letter has been moved to Flashlight in the ASR Application.

The below instructions only apply to wav2letter v0.2, which can be built with Flashlight v0.2 as a dependency.

---

Table of Contents

Installation

• Dependencies
  
• Build Instructions
  
• Running With Docker
  

Training

• Data Preparation
  
• Writing Architecture Files
  
• Train A Model
  
• Distributed Training
  

Decoding

• Beam Search Decoder
  

Python Bindings

• Building python bindings
  
• Python API
  

Inference Framework

• Overview
  
• Tutorial

from https://github.com/flashlight/wav2letter/wiki

------

Build Options

Option	Configuration	Default Value
W2L_BUILD_LIBRARIES_ONLY	ON, OFF	OFF
W2L_LIBRARIES_USE_CUDA	ON, OFF	ON
W2L_LIBRARIES_USE_KENLM	ON, OFF	ON
W2L_LIBRARIES_USE_MKL	ON, OFF	ON
W2L_BUILD_FOR_PYTHON	ON, OFF	OFF
W2L_BUILD_TESTS	ON, OFF	ON
W2L_BUILD_EXAMPLES	ON, OFF	ON
W2L_BUILD_EXPERIMENTAL	ON, OFF	OFF
W2L_BUILD_RECIPES	ON, OFF	ON
W2L_BUILD_SCRIPTS	ON, OFF	OFF
W2L_BUILD_TOOLS	ON, OFF	OFF
CMAKE_BUILD_TYPE		Debug

Inference Pipeline Options

Option	Configuration	Default Value
W2L_BUILD_INFERENCE	ON, OFF	OFF
W2L_INFERENCE_BUILD_TESTS	ON, OFF	ON
W2L_INFERENCE_BUILD_EXAMPLES	ON, OFF	ON
W2L_INFERENCE_BACKEND	[fbgemm,]	fbgemm

General Build Instructions

Once you have the needed dependencies installed, clone the repository:

git clone --recursive https://github.com/facebookresearch/wav2letter.git

and follow the build instructions for your specific OS.

There is no install procedure currently supported for wav2letter++. Building produces three binaries in the build directory:

• Train: given a dataset of input audio and corresponding transcriptions in sub-word units (graphemes, phonemes, etc), trains the acoustic model.
• Test: performs inference on a given dataset with an acoustic model.
• Decode: given an acoustic model/pre-computed network emissions and a language model, computes the most likely sequence of words for a given dataset.

Building on Linux

wav2letter++ has been tested on many Linux distributions including Ubuntu, Debian, CentOS, Amazon Linux, and RHEL.

Assuming you have ArrayFire, flashlight, libsndfile, and KenLM built/installed, install the below dependencies with apt (or your distribution's package manager):

sudo apt-get update
sudo apt-get install \
    # Audio encoding libs for libsndfile \
    libasound2-dev \
    libflac-dev \
    libogg-dev \
    libtool \
    libvorbis-dev \
    # FFTW for Fourier transforms \
    libfftw3-dev \
    # Compression libraries for KenLM \
    zlib1g-dev \
    libbz2-dev \
    liblzma-dev \
    libboost-all-dev \
    # gflags \
    libgflags-dev \
    libgflags2v5 \
    # glog \
    libgoogle-glog-dev \
    libgoogle-glog0v5 \

MKL and KenLM aren't easily discovered by CMake by default; export environment variables to make sure they're found. On most Linux-based systems, MKL is installed in /opt/intel/mkl. Since KenLM doesn't support an install step, after building KenLM, point CMake to wherever you downloaded and built KenLM:

export MKLROOT=/opt/intel/mkl # or path to MKL
export KENLM_ROOT_DIR=[path to KenLM]

Once you've downloaded wav2letter++ and built and installed the required dependencies:

# in your wav2letter++ directory
mkdir -p build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
make -j4 # (or any number of threads)

from https://github.com/flashlight/wav2letter/wiki/General-building-instructions