Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhances and documents recipes #815

Merged

Conversation

ivg
Copy link
Member

@ivg ivg commented Apr 5, 2018

In case if you happen to know already what recipes in bap are, then
the main chainges are summarized below:

  • the recipe is no longer requred to be a zip file, plain files, or
    directories are now accepted

  • the recipes interface is now documented and a couple of issues were
    fixed during the process

  • filenames are no longer substituted, however there is now a builtin
    parameter $prefix that is substituted with the path to the recipe
    root folder.

For the rest, I'm copypasting excerpts from the bap man page, that
describe the recipe system in detail as well as few motivational
examples

RECIPE DESCRIPTION
       A recipe is either a single file or a directory (optionally zipped)
       that contains a parametrized specification of command line parameters
       and support files if necessary.

       The purpose of the recipe is to make bap runs reproducible, so that you
       can share one simple file - the recipe, instead of a bunch of scripts
       together with some verbal instructions. Since recipes introduce an
       extra layer of abstraction they actually simplify interaction with bap
       by hiding unnecessary details. Recipes also provide a mechanism for
       building ready solutions and sharing them with users.

       To use a recipe, just specify its name using the --recipe command line
       parameter. If a recipe has parameters then they could be specified as
       colon separated list of <key>=<value> pairs. See the --recipe parameter
       for more information. To read the recipe description, including the
       list of parameters and resulting command line, use the --show-recipe
       command line option. To list all installed recipes use the
       --list-recipes option.

       The main (and the only necessary) part of a recipe is the recipe
       specification, that is a file that contains a list of recipe items in
       an arbitrary order. Each item is either a command line option, a
       parameter, or a reference to another recipe. All items share the same
       syntax - they are flat s-expressions, i.e., a whitespace separated list
       of strings enclosed in parentheses. The first string in the lists
       denotes the type of the item. E.g., (option run-entry-points malloc
       calloc free).

       The option command requires one mandatory parameter, the option name,
       and an arbitrary number of arguments that will be passed to the
       corresponding command line option. If there are more than one argument
       then they will be concatenated with the comman symbol, e.g., (option
       opt a b c d) will be translated to --opt=a,b,c,d. Option arguments may
       contain substitution symbols. A subsitution symbol starts with the
       dollar sign, that is followed by a named (optionally delimited with
       curly braces, to disambiguate it from the rest of the argument). There
       is one built in parameter called prefix, that is substituted with the
       path to the recipe top folder. See the parameter command to learn how
       to introduce parameters.

       The parameter command introduces a parameter to the recipe, i.e., a
       variable ingredient that could be changed when the recipe is used. The
       parameter command has 3 arguments, all required. The first argument is
       the parameter name, the second is the default value, that is used if
       the a parameter wasn't set, and the last argument is the parameter
       description. The substitution symbol will be replaced with the default
       value of a parameter, if a value of the parameter wasn't passed through
       the command line. Example,

           (parameter depth 128 "maximum depth of analysis")
           (option analysis-depth depth)

       If the parameter is not set through the command line, then it will be
       substituted with 128 otherwise it will receive whatever value a user
       has passed. Finally, the extend command is like the include statement
       in the C preprocessor as it includes all the ingredients from another
       recipe. (Make sure that you're not introducing loops!). The command has
       one mandatory argument, the name of the recipe to include.

RECIPE GRAMMAR
           recipe ::= {<recipe-item>}
           recipe-item ::= <option> | <parameter> | <extend>
           option ::= (option <atom> {<atom>})
           parameter ::= (parameter <atom> <atom> <atom>)
           extend ::= (extend <atom>)

And here comes the description of the --recipe command line
parameter:

       --recipe=VAL
           Load the specified recipe. The  parameter specifies the name of the
           recipe along with an optional colon separated list of arguments.
           The  parameter has the following syntax: <name> : <arg1>=<val1> :
           <arg2>=<val2> : .... The <name> part could be either a path to a
           valid recipe or the name of a recipe, that would be search in the
           recipe search paths. A name may omit .recipe or .scm extensions for
           brevity. The valid representations of a recipe is either the recipe
           file itself (i.e., a file consisting of a list of s-expressions), a
           directory with a valid undefined file, or zip file, that contains a
           valid recipe directory. See the RECIPES section for more
           information. If a recipe is a zip file then it will be unpacked to
           a temporary folder that will be removed after the program
           terminates. Otherwise, all recipe resources will be used as is and
           won't be restored to their previous state, e.g., if the
           input/output files were provided, and they were modified, the
           modifications will persist.

This is how the --list-recipes option works:

$ bap --list-recipe
uaf                              Performs Use-After-Free Analysis
fuzz                             Testing Recipe
derive-inline                    Description was not provided
direct                           Testing Recipe

And this is how a description of the recipe is printed:

$ bap --show-recipe uaf
DESCRIPTION

Performs Use-After-Free Analysis

Uses the Primus promiscuous mode and fuzzes a binary through the specified
$entry points with the maximum depth set to $depth RTL instructions. Outputs
all found Use-After-Free incidents into the `incident` file.

PARAMETERS

- depth - maximum number of instructions per path (defaults to 8192)
- width - maximum number of visits to the same block (defaults to 128)
- entry - entry points (defaults to all-subroutines)

COMMAND LINE

--run \
--run-entry-points=all-subroutines \
--primus-greedy-scheduler \
--primus-limit-max-length=8192 \
--primus-limit-max-visited=128 \
--primus-promiscuous-mode \
--primus-lisp-load=posix,memcheck-malloc,limit-malloc \
--primus-print-obs=incident,incident-location,call-return \
--primus-print-rules=/tmp/recipe-943c0231-491a-43a5-86de-e6c2c662396f.unzipped/select.scm \
--primus-print-output=incidents \
--primus-lisp-channel-redirect=<stdin>:/tmp/recipe-943c0231-491a-43a5-86de-e6c2c662396f.unzipped/stdin,<stdout>:/tmp/recipe-943c0231-491a-43a5-86de-e6c2c662396f.unzipped/stdout \
--report-progress \
--log-dir=log

In case if you happen to know already what recipes in bap are, then
the main chainges are summarized below:

- the recipe is no longer requred to be a zip file, plain files, or
  directories are now accepted

- the recipes interface is now documented and a couple of issues were
  fixed during the process

- filenames are no longer substituted, however there is now a builtin
  parameter `$prefix` that is substituted with the path to the recipe
  root folder.

For the rest, I'm copypasting excerpts from the bap man page, that
describe the recipe system in detail as well as few motivational
examples

```
RECIPE DESCRIPTION
       A recipe is either a single file or a directory (optionally zipped)
       that contains a parametrized specification of command line parameters
       and support files if necessary.

       The purpose of the recipe is to make bap runs reproducible, so that you
       can share one simple file - the recipe, instead of a bunch of scripts
       together with some verbal instructions. Since recipes introduce an
       extra layer of abstraction they actually simplify interaction with bap
       by hiding unnecessary details. Recipes also provide a mechanism for
       building ready solutions and sharing them with users.

       To use a recipe, just specify its name using the --recipe command line
       parameter. If a recipe has parameters then they could be specified as
       colon separated list of <key>=<value> pairs. See the --recipe parameter
       for more information. To read the recipe description, including the
       list of parameters and resulting command line, use the --show-recipe
       command line option. To list all installed recipes use the
       --list-recipes option.

       The main (and the only necessary) part of a recipe is the recipe
       specification, that is a file that contains a list of recipe items in
       an arbitrary order. Each item is either a command line option, a
       parameter, or a reference to another recipe. All items share the same
       syntax - they are flat s-expressions, i.e., a whitespace separated list
       of strings enclosed in parentheses. The first string in the lists
       denotes the type of the item. E.g., (option run-entry-points malloc
       calloc free).

       The option command requires one mandatory parameter, the option name,
       and an arbitrary number of arguments that will be passed to the
       corresponding command line option. If there are more than one argument
       then they will be concatenated with the comman symbol, e.g., (option
       opt a b c d) will be translated to --opt=a,b,c,d. Option arguments may
       contain substitution symbols. A subsitution symbol starts with the
       dollar sign, that is followed by a named (optionally delimited with
       curly braces, to disambiguate it from the rest of the argument). There
       is one built in parameter called prefix, that is substituted with the
       path to the recipe top folder. See the parameter command to learn how
       to introduce parameters.

       The parameter command introduces a parameter to the recipe, i.e., a
       variable ingredient that could be changed when the recipe is used. The
       parameter command has 3 arguments, all required. The first argument is
       the parameter name, the second is the default value, that is used if
       the a parameter wasn't set, and the last argument is the parameter
       description. The substitution symbol will be replaced with the default
       value of a parameter, if a value of the parameter wasn't passed through
       the command line. Example,

           (parameter depth 128 "maximum depth of analysis")
           (option analysis-depth depth)

       If the parameter is not set through the command line, then it will be
       substituted with 128 otherwise it will receive whatever value a user
       has passed. Finally, the extend command is like the include statement
       in the C preprocessor as it includes all the ingredients from another
       recipe. (Make sure that you're not introducing loops!). The command has
       one mandatory argument, the name of the recipe to include.

RECIPE GRAMMAR
           recipe ::= {<recipe-item>}
           recipe-item ::= <option> | <parameter> | <extend>
           option ::= (option <atom> {<atom>})
           parameter ::= (parameter <atom> <atom> <atom>)
           extend ::= (extend <atom>)
```

And here comes the description of the `--recipe` command line
parameter:

```
       --recipe=VAL
           Load the specified recipe. The  parameter specifies the name of the
           recipe along with an optional colon separated list of arguments.
           The  parameter has the following syntax: <name> : <arg1>=<val1> :
           <arg2>=<val2> : .... The <name> part could be either a path to a
           valid recipe or the name of a recipe, that would be search in the
           recipe search paths. A name may omit .recipe or .scm extensions for
           brevity. The valid representations of a recipe is either the recipe
           file itself (i.e., a file consisting of a list of s-expressions), a
           directory with a valid undefined file, or zip file, that contains a
           valid recipe directory. See the RECIPES section for more
           information. If a recipe is a zip file then it will be unpacked to
           a temporary folder that will be removed after the program
           terminates. Otherwise, all recipe resources will be used as is and
           won't be restored to their previous state, e.g., if the
           input/output files were provided, and they were modified, the
           modifications will persist.

```

This is how the `--list-recipes` option works:

```
$ bap --list-recipe
uaf                              Performs Use-After-Free Analysis
fuzz                             Testing Recipe
derive-inline                    Description was not provided
direct                           Testing Recipe
```

And this is how a description of the recipe is printed:

```
$ bap --show-recipe uaf
DESCRIPTION

Performs Use-After-Free Analysis

Uses the Primus promiscuous mode and fuzzes a binary through the specified
$entry points with the maximum depth set to $depth RTL instructions. Outputs
all found Use-After-Free incidents into the `incident` file.

PARAMETERS

- depth - maximum number of instructions per path (defaults to 8192)
- width - maximum number of visits to the same block (defaults to 128)
- entry - entry points (defaults to all-subroutines)

COMMAND LINE

--run \
--run-entry-points=all-subroutines \
--primus-greedy-scheduler \
--primus-limit-max-length=8192 \
--primus-limit-max-visited=128 \
--primus-promiscuous-mode \
--primus-lisp-load=posix,memcheck-malloc,limit-malloc \
--primus-print-obs=incident,incident-location,call-return \
--primus-print-rules=/tmp/recipe-943c0231-491a-43a5-86de-e6c2c662396f.unzipped/select.scm \
--primus-print-output=incidents \
--primus-lisp-channel-redirect=<stdin>:/tmp/recipe-943c0231-491a-43a5-86de-e6c2c662396f.unzipped/stdin,<stdout>:/tmp/recipe-943c0231-491a-43a5-86de-e6c2c662396f.unzipped/stdout \
--report-progress \
--log-dir=log
```
@ivg ivg requested a review from gitoleg April 5, 2018 19:13
| Some tmp -> tmp in
let path =
String.concat [tmp; Filename.dir_sep; prefix; name; suffix] in
match Unix.mkdir path mode with
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will mkdir create directories recursively?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it won't. Neither it should.

@ivg ivg merged commit 293ad10 into BinaryAnalysisPlatform:master Apr 9, 2018
@ivg ivg deleted the enchances-and-documents-recipes branch September 13, 2018 12:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants