haskell-te: 64a61dfb1c33e52f0f541b032be999049c6450e9
1: From: Chris Warburton
2: Date: Tue, 01 Aug 2017 17:29:08 +0100
3: State: resolved
4: Subject: Tease out independent components again.
5: Message-Id: <bdc1e1f920a1ecfa-0-artemis@nixos>
6: resolution: fixed
7:
8: This began as a collection of bash scripts (oh how times have changed!),
9: which were at least runnable as standalone commands.
10:
11: The closest we have right now are the quickspecBench, hashspecBench and
12: mlspecBench commands, which are awful for the following reasons:
13:
14: - Their primary purpose is to benchmark these systems, rather than just
15: run them.
16: - They're FAR too complicated, mostly because of the benchmarking
17: mentioned above.
18: - They try to do too much, for example they include one mode which
19: samples from TEB and one mode which takes TIP from stdin. They
20: include TIP to Haskell package translation (from TEB). And so on.
21: - Their functionality is reasonably modular, but it's invoked in weird
22: ways (e.g. calling out to nix-build).
23:
24: A lot of this is due to the requirements of benchmarking: we need to
25: repeat one bunch of work (the exploration) without repeating other
26: expensive work (like AST extraction); we need to be flexible enough to
27: support arbitrary packages, whilst ensuring the benchmarked sections
28: don't invoke any compilers; and so on.
29:
30: Using asv lets us offload a bunch of these concerns:
31:
32: - Benchmarking and repetition are handled by our Python benchmark
33: scripts.
34: - Preparing the input (translation, etc.) can be taken care of by Nix
35: and/or Python.
36: - Compiling, setting up an appropriate environment, etc. can be taken
37: care of by Nix, in benchmarks/default.nix.
38:
39: We should try, as much as possible, to turn the actual exploration
40: systems into reasonably-standalone scripts. These should probably be
41: something like:
42:
43: - A TIP to haskell package script (taken from TEB)
44: - A Haskell package to ASTs script
45: - A TIP sample to ASTs script
46: - A script to explore ASTs with QuickSpec (no reduction)
47: - Ditto for HashSpec (no reduction)
48: - Ditto for MLSpec (no reduction)
49:
50: These should, as far as possible, avoid doing stuff like caching to the
51: Nix store. They don't need to, and they're modular enough that we can do
52: so in our benchmark environments.
Generated by git2html.