Preparation slide (The presentation starts from the next slide. Just left-click to start.)

Change the Presentation Settings




ディスプレイの調整 〜 Precision M6500の場合

Hands-on Tutorial for Hacking MagicHaskeller
(AAIP 2015 @ Dagstuhl)

Susumu Katayama 片山 晋

October, 2015 2015年10月9日

Before we start

Start installing required programs now, if you haven't.

  1. Choose your OS
    • Native OS of your notebook
      • Linux ... Good
      • Windows ... Limited, but not so bad
      • MacOS, etc. ... Unknown, should work
    • Or log on to your Linux desktop remotely.

      This is another good option because we will not use GUI.

  2. Haskell Platform

    Please install the 2014 or 2015 version.

  3. Unix: Add export PATH=$HOME/.cabal/bin/:$PATH to .bashrc or so.

    Windows: Add %APPDATA%\cabal\bin to Path. (But this may be unnecessary when using the 2015 version of Haskell Platform.)

  4. MagicHaskeller
    $ cabal update $ cabal install -j MagicHaskeller

    On Windows + Platform 2014 you may need to use
    $ cabal install -j MagicHaskeller --flags="-NETWORKURI"

    The point is to force to use the already-installed version of the network package.

    You may need to unregister some package temporarily, and restart installation again.

    In my case, I had to unregister the HTTP package, so
    $ ghc-pkg unregister HTTP $ cabal install -j MagicHaskeller

How you can quickly find this documentation

This tutorial

  1. Installation
  2. Short intro to MagicHaskeller
  3. Short intro to Haskell
  4. Components of package MagicHaskeller
  5. Interactive (standalone) mode of the backend server
  6. Hosting a server system
  7. Customizing the backend server
  8. Using the library from GHCi

What is MagicHaskeller

MagicHaskeller on the Web
Inductive functional programming system / service which is

(Local copy of my home page, with links to the locally-running MagicHaskeller, just in case when the network is down.)

Reference: (Subset of) Haskell's Grammar Haskellの予備知識:文法(のサブセット)

expr ::= variable sequence of alphanumeric letters (or _ or ') starting with a small letter 小文字から始まる英数字(と_と')の列。
e.g. a, x, map, filter
| constructor sequence of alphanumeric letters (or _ or ') starting with a capital letter 大文字から始まる英数字(と_と')の列。
e.g. Nothing
| literal e.g. 0, 'c', "string"
| expr expr
curried function application
関数適用. (引数が複数ならカリー化)
| \ pattern ... pattern -> expr
lambda abstraction
| expr op expr e.g. 3 + 4 / 5
There are different precedences 演算子ごとに優先順位あり
| section e.g.
(*) (equivalent to \x y -> x*y),
(4-) (equivalent to \x -> 4-x),
(+3) (equivalent to \x -> x+3)
(though (-2) is "minus two" as in mathematics)
(4-) (\x -> 4-xと同じ),
(+3) (\x -> x+3と同じ)
| ( expr , ... , expr )
tuple. Note that the parentheses are mandatory.
() is the only value in the unit type.
tuple. カッコは必須
()は unit型()の唯一の要素.
| if expr then expr else expr conditional条件式。見ての通り。
| [ expr , ... , expr ]
enumerated list. The empty list is []
enumerated list. The empty list is []
| [ expr .. expr ] e.g. [1..5] == [1,2,3,4,5] and ['a'..'c'] == ['a','b','c'] == "abc"
| expr :: ctxttype type signature

op ::= varop string of symbols not starting with ::以外から始まる記号の列。e.g. *, ++
| conop string of symbols starting with ::から始まる記号の列。e.g. :, :+

pattern ::= variable bind the variable to the value変数パターン
| _ ignore the value (wild card) ワイルドカード。結果は捨てられる。

ctxttype ::= ctxt => type type with context (constraint by type class) コンテキスト(型クラスによる制約)つき型
| type type without context コンテキストのない型

type ::= variable type variable 型変数
| constructor type constructor 型コンストラクタ
| type type type application 適用
| type -> type function type (left associative)関数の型。左結合。
| ( type , ... , type ) tuple, e.g., (Int,Char).
The unit type is ().
| [ type ] list, e.g., [Integer]

Reference: Standard Types of Haskell Haskellの予備知識:標準的な型

bounded integer範囲付き整数
unbounded integer範囲なし整数.GNU MP Libraryで実装.
list of T'sTのリスト
string. String = [Char] and "abc" == ['a','b','c'] 文字列.実は String = [Char]で,"abc" == ['a','b','c']
(T1, ..., Tn )
tuple of T1, ..., Tn T1, ..., Tnの組
Float, Double
floating point numbers 浮動小数点数.
IO action returning T (value with side effect). MagicHaskeller does not deal with IO. Tを返すIO action (副作用のある動作).MagicHaskellerは現在のところ扱わない.

Reference: Haskell's Type Classes Haskellの予備知識:型クラス

type class
collection of methods, corresponding to interface of Java
type T is a member of type class C
the methods of type class C are defined for type T.
class Eq a where -- Eq a <=> relation (==) :: a -> a -> Bool -- (==) :: Eq a => a -> a -> Bool -- is defined data Nat = Zero | Succ Nat -- Take Nat as an example. (This is not in the library.)
例: class Eq a where -- Eq a <=> 型a上に関係 (==) :: a -> a -> Bool -- (==) :: Eq a => a -> a -> Bool -- が定義される. data Nat = Zero | Succ Nat -- 自然数を例にとる.ライブラリにはない.
instance Eq Nat where Zero == Zero = True Zero == Succ n = False Succ m == Zero = False Succ m == Succ n = m == n

Math operators are defined using type classes.


Numeric literals are also defined using type classes.


Integer literals
have type Num a => a,
and are converted using fromInteger :: Num a => Integer -> a which is a method of Num
Fractional literals
have type Fractional a => a,
and are converted using fromRational :: Fractional a => Rational -> a which is a method of Fractional
Num a => aという型を持つ.NumのメソッドであるfromInteger :: Num a => Integer -> aを使って型変換
Fractional a => aという型を持つ.FractionalのメソッドであるfromRational :: Fractional a => Rational -> aを使って型変換

Components of package MagicHaskeller MagicHaskellerパッケージに含まれるもの

memoization table メモテーブル
Package description
The (aged) library API.
Executable MagicHaskeller
The backend server. This can be used as an interactive program.
Executable MagicHaskeller.cgi
The CGI frontend.

Executable MagicHaskeller

Just try these (not at the root of the source directory):

$ MagicHaskeller -?
$ MagicHaskeller -d 5 -i
Interactive mode, synthesize expressions with at most 4 function applications, without using fractional or rational numbers.
$ MagicHaskeller --excel -d 7 -i
Interactive mode, synthesize using Excel functions (MagicExceller)
Ctrl-d to exit.

And try this if your notebook has more than 64GB memory (same amount as our backend server in Japan):

$ MagicHaskeller -i -d 7 -2 +RTS -N8
Interactive mode, synthesize expressions with at most 6 function applications, using Doubles in addition to other types, using 8 cores.
+RTS indicates that the following options are passed to the GHC runtime system. Use +RTS -? for available RTS options.


By default, training starts at the startup, using ./predicates.
$ MagicHaskeller -d 5 -t predicates -i
Queries in predicates are automatically supplied in parallel at the background.
$ MagicHaskeller -d 5 -n -t predicates -s predicatesServed -i
Queries in predicates are automatically supplied in parallel, then, queries in predicatesServed are supplied sequentially, then the interactive mode starts.
Sample training sets are installed in -s is useful for evaluating the speed rather than for real training.

Running Server Systems

$ nohup MagicHaskeller -d 7 -2 -n -t predicates -s predicatesServed -p 55443 +RTS -N8 -RTS &
MagicHaskeller backend server running in Japan, using the port 55443
$ nohup MagicHaskeller --excel -d 8 -p 55444 +RTS -N4 -RTS &
MagicExceller backend server running in Japan, using the port 55444

The CGI frontend

  1. Install a web server with CGI support.
  2. Install MagicHaskeller globally.
  3. Copy /usr/local/bin/MagicHaskeller to the CGI directory.
  4. Copy there and edit /usr/local/share/MagicHaskeller-

So, the installation command for the CGI server is: $ cabal install MagicHaskeller --with-compiler=ghc-7.4.1 --global --root-cmd=sudo (Adapt the GHC version number.)

CGI settings

Looking into / Maybe Customizing the Backend Server

In order to freely customize the component library (or the set of functions with which to synthesize), you have to hack the source.

Getting the source

mkdir foo cd foo cabal unpack MagicHaskeller (Just execute cabal install (without the package name) to build and install the source tree under the current directory.)

Important files

defines main for Executable MagicHaskeller
imports MagicHaskeller/LibTH.hs
maybe bloating planlessly
defines some component libraries and the postprocessor.
imports MagicHaskeller.lhs
bloated planlessly
the default component library without Double or Rational
the component library with Double, used by MagicHaskeller -2
the component library with Ratio, used by MagicHaskeller -w
the component library with Double and Ratio, used by MagicHaskeller -b
defines API for synthesis and defining component libraries.
designed planlessly

Internals of LibTH.hs


  • some useful combinations of functions (e.g. \f x n -> iterate f x !! abs n) are considered 1 function
    いくつかの有用な関数の組み合わせ(たとえば \f x n -> iterate f x !! abs n)は1つの関数と数える。
  • uncommon functions (e.g. gcd) with less priority are considered to involve 1 or 2 applications
    あまり使われない関数 (たとえば gcd) は、優先順序を下げて、関数適用が1〜2回入っていると考えて数える。
  • Using the Library API from GHCi

    You can execute synthesis without rebuilding executables, from GHCi (or GHC interactive).
    $ ghci -fth -package MagicHaskeller Prelude> :m +MagicHaskeller.LibTH Prelude MagicHaskeller.LibTH> filterThen (\f -> f [1,2,3] == [3,3,3]) (everything pgfull False) >>= pprs . take 6 Prelude MagicHaskeller.LibTH> let mypg = mkPG $(p [| (map :: (a -> b) -> (->) [a] [b], show :: Integer -> [Char]) |]) :: ProgGenSF Prelude MagicHaskeller.LibTH> filterThen (\f -> f [1,2,3] == ["1","2","3"]) (everything mypg False) >>= pprs . take 7 Prelude MagicHaskeller.LibTH> :m +MagicHaskeller.Minimal Prelude MagicHaskeller.LibTH MagicHaskeller.Minimal> map pprint $ concat $ take 6 $ f1EF postprocess (\f -> f [1,2,3::Int] == "123") pgfull False
    See the library documentation (mainly of module MagicHaskeller) for details.


    Other tips for efficiency: