Support #604
Updated by Daniel Curtis about 9 years ago
This is a guide for installing the intelligent personal assistant Sirius on Ubuntu 14.04 minimal. h2. Prepare the environment * Make sure the system is up to date: <pre> sudo -s apt-get update && apt-get upgrade </pre> * Install wget: <pre> apt-get install wget </pre> h2. Install the Sirius Application *NOTE*: Sirius and its dependencies is several gigabytes, make sure to allocate enough storage space. * Clone sirius from github: <pre> git clone https://github.com/jhauswald/sirius.git cd sirius/sirius-application tar xzf question-answer.tar.gz </pre> * Add additional repositories for ffmpeg <pre> add-apt-repository ppa:kirillshkrogalev/ffmpeg-next </pre> * Enable multiverse sources for libfaac-dev <pre> apt-add-repository multiverse </pre> * Update sources and install basic dependencies <pre> apt-get update apt-get install git zip unzip subversion sox default-jdk ant automake autoconf libtool bison libboost-all-dev ffmpeg swig python-pip curl </pre> * Install opencv dependencies <pre> apt-get install build-essential checkinstall git cmake libfaac-dev libjack-jackd2-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libsdl1.2-dev libtheora-dev libva-dev libvdpau-dev libvorbis-dev libx11-dev libxfixes-dev libxvidcore-dev texi2html yasm zlib1g-dev </pre> * Install tessaract text recognition <pre> apt-get install tesseract-ocr tesseract-ocr-eng libtesseract-dev libleptonica-dev </pre> * Install ATLAS library for Kaldi <pre> apt-get install libatlas-dev libatlas-base-dev </pre> * Install protobuf for image-matching <pre> apt-get install libprotobuf-dev protobuf-compiler </pre> * Install dependencies for the web application <pre> pip install wtforms Flask requests pickledb </pre> h3. Install opencv * Clone opencv from github <pre> git clone https://github.com/Itseez/opencv.git opencv-2.4.9 cd opencv-2.4.9 git checkout 2.4.9 </pre> * Build and install opencv <pre> mkdir build cd build cmake .. make -j4 make -j4 install ldconfig -v </pre> * Prepare kaldi <pre> cd ~/sirius/sirius-application/speech-recognition/kaldi/scripts ./prepare.sh </pre> * Compile Sirius <pre> cd ~/sirius/sirius-application ./compile-sirius-servers.sh </pre> h2. Running Sirius h3. Automatic Speech Recognition (ASR) Sirius supports three backends: +Kaldi+ (DNN/HMM based), +Pocketsphinx+, and +Sphinx4+ (the latter are GMM/HMM based) *NOTE*: The wikipedia archive is around 11GB compressed, make sure to perform Automatic Speech Recognition. allocate enough storage space. * To open an ASR server: <pre> cd ~/sirius/sirius-application/run-scripts ./start-asr-server.sh </pre> #* or use the pocketsphinx ASR <pre> ./start-asr-server.sh pocketsphinx </pre> #* or specify an ASR, hostname and port: <pre> ./start-asr-server.sh pocketsphinx localhost 8080 </pre> * In a separate terminal, test the ASR: <pre> ./sirius-asr-test.sh ../inputs/questions/what.is.the.speed.of.light.wav </pre> h3. Image Matching (IMM) Image Matching uses SURF to match query images to a stored database. * In image-matching/ first build and store a database of descriptors in protobuf format where the arguments are the name of the database and the directory containing the images: <pre> cd ~/sirius/sirius-application/image-matching ./make-db.py landmarks matching/landmarks/db/ </pre> To change the database used by the IMM service, change the name in start-imm-server.py. * In run-scripts/, open the IMM server: <pre> cd ~/sirius/sirius-application/run-scripts ./start-imm-server.sh </pre> * In a separate terminal, test IMM using: <pre> ./sirius-imm-test.sh ../image-matching/matching/landmarks/query/query.jpg </pre> h3. Question-Answering System (QA) The Question-Answering system uses OpenEphyra and a Wikipedia database stored in Lemur’s Indri format. * Extract the Wikipedia database (after untaring and building question-answer): <pre> cd ~/sirius/sirius-application wget http://web.eecs.umich.edu/~jahausw/download/wiki_indri_index.tar.gz tar xzvf wiki_indri_index.tar.gz -C question-answer/ </pre> * In run-scripts/, open the QA server: <pre> ./start-qa-server.sh </pre> * In a separate terminal, test QA using: <pre> ./sirius-qa-test.sh "what is the speed of light" </pre> h3. Combining the Services * It is very easy with Sirius to combine ASR and QA to create the full intelligent personal assistant pipeline. After opening multiple servers using ./start-<service>-server.sh, test an ASR-QA query using: <pre> ./sirius-asr-qa-test.sh ../inputs/real/what.is.the.capital.of.italy.wav </pre> h2. Resources * http://sirius.clarity-lab.org/sirius/ * https://github.com/jhauswald/sirius