nix-config: 5b4a3a7baa888288c691cc953d95cf7972dc9167

     1: From: Chris Warburton
     2: Date: Tue, 15 May 2018 17:47:54 +0100
     3: Subject: Re: Only test things which aren't derivations
     4: Message-Id: <862c9e42944a96aa-de2ebeeee3868c69-artemis@nixos>
     5: References: <862c9e42944a96aa-0-artemis@nixos>
     6: In-Reply-To: <862c9e42944a96aa-0-artemis@nixos>
     7: 
     8: I've now taken a different approach: each file can define both a 'pkg'
     9: (which could actually be a helper function, or whatever) *and* a list of
    10: tests. These lists are all concatenated into a 'customTests' list,
    11: similar to how `customPkgNames` accumulates names.
    12: 
    13: The idea is as follows:
    14: 
    15:  - Tests which should checked as soon as we *use* a definition should be
    16:    assertions on those definitions. For example, checking that some
    17:    dummy input gives the right sort of thing. These are like unit tests,
    18:    since they're quick sanity checks of the functionality.
    19:  - More thorough tests, which are slow but important, should be
    20:    derivations which the package depends on (e.g. using 'withDeps').
    21:    Of course, this only applies to derivations; however it can also be
    22:    used by functions which produce derivations, since they can make
    23:    their output depend on the tests passing.
    24:  - Assertions can be a bad idea for some things, since we might not want
    25:    to abort all invocations when they fail. These can be moved on to the
    26:    'tests' list.
    27:  - If there's a heavyweight test, like integration tests, which we might
    28:    not want to force all users to depend on, we can make them
    29:    derivations and put them in the 'tests' list. Examples might be
    30:    checking that our Haskell helper functions work, e.g.
    31:    'haskellPkgWithDeps' and friends.