Welcome to YARA’s documentation!¶

Installing on Windows¶

Compiled binaries for Windows in both 32 and 64 bits flavors can be found in the link below. Just download the version of you want, unzip the archive, and put the yara.exe and yarac.exe binaries anywhere in your disk.

To install the yara-python extension download an execute the installer corresponding to the version of Python you’re using.

Download Windows binaries

If you want to build YARA yourself you can use the Visual Studio 2010 or Visual Studio 2015 projects found in the source tree under ./windows/vs2010 and ./windows/vs2015 respectively.

Installing on Mac OS X with Homebrew¶

To install YARA using Homebrew simply type brew install yara.

Installing yara-python¶

If you plan to use YARA from your Python scripts you need to install the yara-python extension. Please refer to https://github.com/VirusTotal/yara-python for instructions on how to install it.

Hexadecimal strings¶

Hexadecimal strings allow three special constructions that make them more flexible: wild-cards, jumps, and alternatives. Wild-cards are just placeholders that you can put into the string indicating that some bytes are unknown and they should match anything. The placeholder character is the question mark (?). Here you have an example of a hexadecimal string with wild-cards:

rule WildcardExample
{
    strings:
       $hex_string = { E2 34 ?? C8 A? FB }

    condition:
       $hex_string
}

As shown in the example the wild-cards are nibble-wise, which means that you can define just one nibble of the byte and leave the other unknown.

Wild-cards are useful when defining strings whose content can vary but you know the length of the variable chunks, however, this is not always the case. In some circumstances you may need to define strings with chunks of variable content and length. In those situations you can use jumps instead of wild-cards:

rule JumpExample
{
        strings:
           $hex_string = { F4 23 [4-6] 62 B4 }

        condition:
           $hex_string
}

In the example above we have a pair of numbers enclosed in square brackets and separated by a hyphen, that’s a jump. This jump is indicating that any arbitrary sequence from 4 to 6 bytes can occupy the position of the jump. Any of the following strings will match the pattern:

F4 23 01 02 03 04 62 B4
F4 23 00 00 00 00 00 62 B4
F4 23 15 82 A3 04 45 22 62 B4

Any jump [X-Y] must met the condition 0 <= X <= Y. In previous versions of YARA both X and Y must be lower than 256, but starting with YARA 2.0 there is no limit for X and Y.

These are valid jumps:

FE 39 45 [0-8] 89 00
FE 39 45 [23-45] 89 00
FE 39 45 [1000-2000] 89 00

This is invalid:

FE 39 45 [10-7] 89 00

If the lower and higher bounds are equal you can write a single number enclosed in brackets, like this:

FE 39 45 [6] 89 00

The above string is equivalent to both of these:

FE 39 45 [6-6] 89 00
FE 39 45 ?? ?? ?? ?? ?? ?? 89 00

Starting with YARA 2.0 you can also use unbounded jumps:

FE 39 45 [10-] 89 00
FE 39 45 [-] 89 00

The first one means [10-infinite], the second one means [0-infinite].

There are also situations in which you may want to provide different alternatives for a given fragment of your hex string. In those situations you can use a syntax which resembles a regular expression:

rule AlternativesExample1
{
    strings:
       $hex_string = { F4 23 ( 62 B4 | 56 ) 45 }

    condition:
       $hex_string
}

This rule will match any file containing F42362B445 or F4235645.

But more than two alternatives can be also expressed. In fact, there are no limits to the amount of alternative sequences you can provide, and neither to their lengths.

rule AlternativesExample2
{
    strings:
       $hex_string = { F4 23 ( 62 B4 | 56 | 45 ?? 67 ) 45 }

    condition:
       $hex_string
}

As can be seen also in the above example, strings containing wild-cards are allowed as part of alternative sequences.

Text strings¶

As shown in previous sections, text strings are generally defined like this:

rule TextExample
{
    strings:
        $text_string = "foobar"

    condition:
       $text_string
}

This is the simplest case: an ASCII-encoded, case-sensitive string. However, text strings can be accompanied by some useful modifiers that alter the way in which the string will be interpreted. Those modifiers are appended at the end of the string definition separated by spaces, as will be discussed below.

Text strings can also contain the following subset of the escape sequences available in the C language:

rule CaseInsensitiveTextExample
{
    strings:
        $text_string = "foobar" nocase

    condition:
        $text_string
}

rule WideCharTextExample1
{
    strings:
        $wide_string = "Borland" wide

    condition:
       $wide_string
}

rule WideCharTextExample2
{
    strings:
        $wide_and_ascii_string = "Borland" wide ascii

    condition:
       $wide_and_ascii_string
}

Regular expressions¶

Regular expressions are one of the most powerful features of YARA. They are defined in the same way as text strings, but enclosed in forward slashes instead of double-quotes, like in the Perl programming language.

rule RegExpExample1
{
    strings:
        $re1 = /md5: [0-9a-zA-Z]{32}/
        $re2 = /state: (on|off)/

    condition:
        $re1 and $re2
}

Regular expressions can be also followed by nocase, ascii, wide, and fullword modifiers just like in text strings. The semantics of these modifiers are the same in both cases.

In previous versions of YARA, external libraries like PCRE and RE2 were used to perform regular expression matching, but starting with version 2.0 YARA uses its own regular expression engine. This new engine implements most features found in PCRE, except a few of them like capture groups, POSIX character classes and backreferences.

YARA’s regular expressions recognise the following metacharacters:

The following quantifiers are recognised as well:

All these quantifiers have a non-greedy variant, followed by a question mark (?):

The following escape sequences are recognised:

These are the recognised character classes:

Starting with version 3.3.0 these zero-with assertions are also recognized:

Counting strings¶

Sometimes we need to know not only if a certain string is present or not, but how many times the string appears in the file or process memory. The number of occurrences of each string is represented by a variable whose name is the string identifier but with a # character in place of the $ character. For example:

rule CountExample
{
    strings:
        $a = "dummy1"
        $b = "dummy2"

    condition:
        #a == 6 and #b > 10
}

This rules matches any file or process containing the string $a exactly six times, and more than ten occurrences of string $b.

String offsets or virtual addresses¶

In the majority of cases, when a string identifier is used in a condition, we are willing to know if the associated string is anywhere within the file or process memory, but sometimes we need to know if the string is at some specific offset on the file or at some virtual address within the process address space. In such situations the operator at is what we need. This operator is used as shown in the following example:

rule AtExample
{
    strings:
        $a = "dummy1"
        $b = "dummy2"

    condition:
        $a at 100 and $b at 200
}

The expression $a at 100 in the above example is true only if string $a is found at offset 100 within the file (or at virtual address 100 if applied to a running process). The string $b should appear at offset 200. Please note that both offsets are decimal, however hexadecimal numbers can be written by adding the prefix 0x before the number as in the C language, which comes very handy when writing virtual addresses. Also note the higher precedence of the operator at over the and.

While the at operator allows to search for a string at some fixed offset in the file or virtual address in a process memory space, the in operator allows to search for the string within a range of offsets or addresses.

rule InExample
{
    strings:
        $a = "dummy1"
        $b = "dummy2"

    condition:
        $a in (0..100) and $b in (100..filesize)
}

In the example above the string $a must be found at an offset between 0 and 100, while string $b must be at an offset between 100 and the end of the file. Again, numbers are decimal by default.

You can also get the offset or virtual address of the i-th occurrence of string $a by using @a[i]. The indexes are one-based, so the first occurrence would be @a[1] the second one @a[2] and so on. If you provide an index greater then the number of occurrences of the string, the result will be a NaN (Not A Number) value.

Match length¶

For many regular expressions and hex strings containing jumps, the length of the match is variable. If you have the regular expression /fo*/ the strings “fo”, “foo” and “fooo” can be matches, all of them with a different length.

You can use the length of the matches as part of your condition by using the character ! in front of the string identifier, in a similar way you use the @ character for the offset. !a[1] is the length for the first match of $a, !a[2] is the length for the second match, and so on. !a is a abbreviated form of !a[1].

File size¶

String identifiers are not the only variables that can appear in a condition (in fact, rules can be defined without any string definition as will be shown below), there are other special variables that can be used as well. One of these especial variables is filesize, which holds, as its name indicates, the size of the file being scanned. The size is expressed in bytes.

rule FileSizeExample
{
    condition:
       filesize > 200KB
}

The previous example also demonstrate the use of the KB postfix. This postfix, when attached to a numerical constant, automatically multiplies the value of the constant by 1024. The MB postfix can be used to multiply the value by 2^20. Both postfixes can be used only with decimal constants.

The use of filesize only makes sense when the rule is applied to a file, if the rule is applied to a running process won’t never match because filesize doesn’t make sense in this context.

Executable entry point¶

Another special variable than can be used on a rule is entrypoint. If file is a Portable Executable (PE) or Executable and Linkable Format (ELF), this variable holds the raw offset of the exectutable’s entry point in case we are scanning a file. If we are scanning a running process, the entrypoint will hold the virtual address of the main executable’s entry point. A typical use of this variable is to look for some pattern at the entry point to detect packers or simple file infectors.

rule EntryPointExample1
{
    strings:
        $a = { E8 00 00 00 00 }

    condition:
       $a at entrypoint
}

rule EntryPointExample2
{
    strings:
        $a = { 9C 50 66 A1 ?? ?? ?? 00 66 A9 ?? ?? 58 0F 85 }

    condition:
       $a in (entrypoint..entrypoint + 10)
}

The presence of the entrypoint variable in a rule implies that only PE or ELF files can satisfy that rule. If the file is not a PE or ELF any rule using this variable evaluates to false.

Accessing data at a given position¶

There are many situations in which you may want to write conditions that depends on data stored at a certain file offset or memory virtual address, depending if we are scanning a file or a running process. In those situations you can use one of the following functions to read data from the file at the given offset:

int8(<offset or virtual address>)
int16(<offset or virtual address>)
int32(<offset or virtual address>)

uint8(<offset or virtual address>)
uint16(<offset or virtual address>)
uint32(<offset or virtual address>)

int8be(<offset or virtual address>)
int16be(<offset or virtual address>)
int32be(<offset or virtual address>)

uint8be(<offset or virtual address>)
uint16be(<offset or virtual address>)
uint32be(<offset or virtual address>)

The intXX functions read 8, 16, and 32 bits signed integers from <offset or virtual address>, while functions uintXX read unsigned integers. Both 16 and 32 bits integer are considered to be little-endian. If you want to read a big-endian integer use the corresponding function ending in be. The <offset or virtual address> parameter can be any expression returning an unsigned integer, including the return value of one the uintXX functions itself. As an example let’s see a rule to distinguish PE files:

rule IsPE
{
  condition:
     // MZ signature at offset 0 and ...
     uint16(0) == 0x5A4D and
     // ... PE signature at offset stored in MZ header at 0x3C
     uint32(uint32(0x3C)) == 0x00004550
}

Sets of strings¶

There are circumstances in which it is necessary to express that the file should contain a certain number strings from a given set. None of the strings in the set are required to be present, but at least some of them should be. In these situations the operator of come into help.

rule OfExample1
{
    strings:
        $a = "dummy1"
        $b = "dummy2"
        $c = "dummy3"

    condition:
        2 of ($a,$b,$c)
}

What this rule says is that at least two of the strings in the set ($a,$b,$c) must be present on the file, no matter which. Of course, when using this operator, the number before the of keyword must be equal to or less than the number of strings in the set.

The elements of the set can be explicitly enumerated like in the previous example, or can be specified by using wild cards. For example:

rule OfExample2
{
    strings:
        $foo1 = "foo1"
        $foo2 = "foo2"
        $foo3 = "foo3"

    condition:
        2 of ($foo*)  // equivalent to 2 of ($foo1,$foo2,$foo3)
}

rule OfExample3
{
    strings:
        $foo1 = "foo1"
        $foo2 = "foo2"

        $bar1 = "bar1"
        $bar2 = "bar2"

    condition:
        3 of ($foo*,$bar1,$bar2)
}

You can even use ($*) to refer to all the strings in your rule, or write the equivalent keyword them for more legibility.

rule OfExample4
{
    strings:
        $a = "dummy1"
        $b = "dummy2"
        $c = "dummy3"

    condition:
        1 of them // equivalent to 1 of ($*)
}

In all the above examples the number of strings have been specified by a numeric constant, but any expression returning a numeric value can be used. The keywords any and all can be used as well.

all of them       // all strings in the rule
any of them       // any string in the rule
all of ($a*)      // all strings whose identifier starts by $a
any of ($a,$b,$c) // any of $a, $b or $c
1 of ($*)         // same that "any of them"

Applying the same condition to many strings¶

There is another operator very similar to of but even more powerful, the for..of operator. The syntax is:

for expression of string_set : ( boolean_expression )

And its meaning is: from those strings in string_set at least expression of them must satisfy boolean_expression.

In other words: boolean_expression is evaluated for every string in string_set and must be at least expression of them returning True.

Of course, boolean_expression can be any boolean expression accepted in the condition section of a rule, except for one important detail: here you can (and should) use a dollar sign ($) as a place-holder for the string being evaluated. Take a look to the following expression:

for any of ($a,$b,$c) : ( $ at entrypoint  )

The $ symbol in the boolean expression is not tied to any particular string, it will be $a, and then $b, and then $c in the three successive evaluations of the expression.

Maybe you already realised that the of operator is an special case of for..of. The following expressions are the same:

any of ($a,$b,$c)
for any of ($a,$b,$c) : ( $ )

You can also employ the symbols # and @ to make reference to the number of occurrences and the first offset of each string respectively.

for all of them : ( # > 3 )
for all of ($a*) : ( @ > @b )

Using anonymous strings with of and for..of¶

When using the of and for..of operators followed by them, the identifier assigned to each string of the rule is usually superfluous. As we are not referencing any string individually we do not need to provide a unique identifier for each of them. In those situations you can declare anonymous strings with identifiers consisting only in the $ character, as in the following example:

rule AnonymousStrings
{
    strings:
        $ = "dummy1"
        $ = "dummy2"

    condition:
        1 of them
}

Iterating over string occurrences¶

As seen in String offsets or virtual addresses, the offsets or virtual addresses where a given string appears within a file or process address space can be accessed by using the syntax: @a[i], where i is an index indicating which occurrence of the string $a you are referring to. (@a[1], @a[2],...).

Sometimes you will need to iterate over some of these offsets and guarantee they satisfy a given condition. For example:

rule Occurrences
{
    strings:
        $a = "dummy1"
        $b = "dummy2"

    condition:
        for all i in (1,2,3) : ( @a[i] + 10 == @b[i] )
}

The previous rule tells that the first three occurrences of $b should be 10 bytes away from the first three occurrences of $a.

The same condition could be written also as:

for all i in (1..3) : ( @a[i] + 10 == @b[i] )

Notice that we’re using a range (1..3) instead of enumerating the index values (1,2,3). Of course, we’re not forced to use constants to specify range boundaries, we can use expressions as well like in the following example:

for all i in (1..#a) : ( @a[i] < 100 )

In this case we’re iterating over every occurrence of $a (remember that #a represents the number of occurrences of $a). This rule is telling that every occurrence of $a should be within the first 100 bytes of the file.

In case you want to express that only some occurrences of the string should satisfy your condition, the same logic seen in the for..of operator applies here:

for any i in (1..#a) : ( @a[i] < 100 )
for 2 i in (1..#a) : ( @a[i] < 100 )

In resume, the syntax of this operator is:

for expression identifier in indexes : ( boolean_expression )

Referencing other rules¶

When writing the condition for a rule you can also make reference to a previously defined rule in a manner that resembles a function invocation of traditional programming languages. In this way you can create rules that depends on others. Let’s see an example:

rule Rule1
{
    strings:
        $a = "dummy1"

    condition:
        $a
}

rule Rule2
{
    strings:
        $a = "dummy2"

    condition:
        $a and Rule1
}

As can be seen in the example, a file will satisfy Rule2 only if it contains the string “dummy2” and satisfy Rule1. Note that is strictly necessary to define the rule being invoked before the one that will make the invocation.

Global rules¶

Global rules give you the possibility of imposing restrictions in all your rules at once. For example, suppose that you want all your rules ignoring those files that exceed certain size limit, you could go rule by rule doing the required modifications to their conditions, or just write a global rule like this one:

global rule SizeLimit
{
    condition:
        filesize < 2MB
}

You can define as many global rules as you want, they will be evaluated before the rest of the rules, which in turn will be evaluated only of all global rules are satisfied.

Private rules¶

Private rules are a very simple concept. They are just rules that are not reported by YARA when they match on a given file. Rules that are not reported at all may seem sterile at first glance, but when mixed with the possibility offered by YARA of referencing one rule from another (see Referencing other rules) they become useful. Private rules can serve as building blocks for other rules, and at the same time prevent cluttering YARA’s output with irrelevant information. For declaring a rule as private just add the keyword private before the rule declaration.

private rule PrivateRuleExample
{
    ...
}

You can apply both private and global modifiers to a rule, resulting a global rule that does not get reported by YARA but must be satisfied.

Rule tags¶

Another useful feature of YARA is the possibility of adding tags to rules. Those tags can be used later to filter YARA’s output and show only the rules that you are interesting in. You can add as many tags as you want to a rule, they are declared after the rule identifier as shown below:

rule TagsExample1 : Foo Bar Baz
{
    ...
}

rule TagsExample2 : Bar
{
    ...
}

Tags must follow the same lexical convention of rule identifiers, therefore only alphanumeric characters and underscores are allowed, and the tag cannot start with a digit. They are also case sensitive.

When using YARA you can output only those rules that are tagged with the tag or tags that you provide.

Metadata¶

Besides the string definition and condition sections, rules can also have a metadata section where you can put additional information about your rule. The metadata section is defined with the keyword meta and contains identifier/value pairs like in the following example:

rule MetadataExample
{
    meta:
        my_identifier_1 = "Some string data"
        my_identifier_2 = 24
        my_identifier_3 = true

    strings:
        $my_text_string = "text here"
        $my_hex_string = { E2 34 A1 C8 23 FB }

    condition:
        $my_text_string or $my_hex_string
}

As can be seen in the example, metadata identifiers are always followed by an equal sign and the value assigned to them. The assigned values can be strings, integers, or one of the boolean values true or false. Note that identifier/value pairs defined in the metadata section can not be used in the condition section, their only purpose is to store additional information about the rule.

Building our “Hello World!”¶

Modules are not magically built into YARA just by dropping their source code into the libyara/modules directory, you must follow two further steps in order to get them to work. The first step is adding your module to the module_list file also found in the libyara/modules directory.

The module_list file looks like this:

MODULE(tests)
MODULE(pe)

#ifdef CUCKOO_MODULE
MODULE(cuckoo)
#endif

You must add a line MODULE(<name>) with the name of your module to this file. In our case the resulting module_list is:

MODULE(tests)
MODULE(pe)

#ifdef CUCKOO_MODULE
MODULE(cuckoo)
#endif

MODULE(demo)

The second step is modifying the Makefile.am to tell the make program that the source code for your module most be compiled and linked into YARA. At the very beginning of libyara/Makefile.am you’ll find this:

MODULES =  modules/tests.c
MODULES += modules/pe.c

if CUCKOO_MODULE
MODULES += modules/cuckoo.c
endif

Just add a new line for your module:

MODULES =  modules/tests.c
MODULES += modules/pe.c

if CUCKOO_MODULE
MODULES += modules/cuckoo.c
endif

MODULES += modules/demo.c

And that’s all! Now you’re ready to build YARA with your brand-new module included. Just go to the source tree root directory and type as always:

make
sudo make install

Now you should be able to create a rule like this:

import "demo"

rule HelloWorld
{
    condition:
        demo.greeting == "Hello World!"
}

Any file scanned with this rule will match the HelloWord because demo.greeting == "Hello World!" is always true.

Basic types¶

Within the declaration section you can use declare_string(<variable name>), declare_integer(<variable name>) and declare_float(<variable name>) to declare string, integer, or float variables respectively. For example:

begin_declarations;

    declare_integer("foo");
    declare_string("bar");
    declare_float("baz");

end_declarations;

Variable names can’t contain characters other than letters, numbers and underscores. These variables can be used later in your rules at any place where an integer or string is expected. Supposing your module name is “mymodule”, they can be used like this:

mymodule.foo > 5

mymodule.bar matches /someregexp/

Structures¶

Your declarations can be organized in a more structured way:

begin_declarations;

    declare_integer("foo");
    declare_string("bar");
    declare_float("baz");

    begin_struct("some_structure");

        declare_integer("foo");

        begin_struct("nested_structure");

            declare_integer("bar");

        end_struct("nested_structure");

    end_struct("some_structure");

    begin_struct("another_structure");

        declare_integer("foo");
        declare_string("bar");
        declare_string("baz");
        declare_float("tux");

    end_struct("another_structure");

end_declarations;

In this example we’re using begin_struct(<structure name>) and end_struct(<structure name>) to delimite two structures named some_structure and another_structure. Within the structure delimiters you can put any other declarations you want, including another structure declaration. Also notice that members of different structures can have the same name, but members within the same structure must have unique names.

When refering to these variables from your rules it would be like this:

mymodule.foo
mymodule.some_structure.foo
mymodule.some_structure.nested_structure.bar
mymodule.another_structure.baz

Arrays¶

In the same way you declare individual strings, integers, floats or structures, you can declare arrays of them:

begin_declarations;

    declare_integer_array("foo");
    declare_string_array("bar");
    declare_float_array("baz");

    begin_struct_array("struct_array");

        declare_integer("foo");
        declare_string("bar");

    end_struct_array("struct_array");

end_declarations;

Individual values in the array are referenced like in most programming languages:

foo[0]
bar[1]
baz[3]
struct_array[4].foo
struct_array[1].bar

Arrays are zero-based and don’t have a fixed size, they will grow as needed when you start initializing its values.

Dictionaries¶

You can also declare dictionaries of integers, floats, strings, or structures:

begin_declarations;

    declare_integer_dictionary("foo");
    declare_string_dictionary("bar");
    declare_float_dictionary("baz")

    begin_struct_dictionary("struct_dict");

        declare_integer("foo");
        declare_string("bar");

    end_struct_dictionary("struct_dict");

end_declarations;

Individual values in the dictionary are accessed by using a string key:

foo["somekey"]
bar["anotherkey"]
baz["yetanotherkey"]
struct_dict["k1"].foo
struct_dict["k1"].bar

Functions¶

One of the more powerful features of YARA modules is the possibility of declaring functions that can be later invoked from your rules. Functions must appear in the declaration section in this way:

declare_function(<function name>, <argument types>, <return tuype>, <C function>);

<function name> is the name that will be used in your YARA rules to invoke the function.

<argument types> is a string containing one character per function argument, where the character indicates the type of the argument. Functions can receive four different types of arguments: string, integer, float and regular expression, denoted by characters: s, i, r and f respectively. If your function receives two integers <argument types> must be “ii”, if it receives an integer as the first argument and a string as the second one <argument types> must be “is”, if it receives three strings and a float <argument types> must be “sssf”.

<return type> is a string with a single character indicating the return type. Possible return types are string (“s”) integer (“i”) and float (“f”).

<C function> is the identifier for the actual implementation of your function.

Here you have a full example:

define_function(isum)
{
  int64_t a = integer_argument(1);
  int64_t b = integer_argument(2);

  return_integer(a + b);
}

define_function(fsum)
{
  double a = float_argument(1);
  double b = float_argument(2);

  return_integer(a + b);
}

begin_declarations;

    declare_function("sum", "ii", "i", sum);

end_declarations;

As you can see in the example above, your function code must be defined before the declaration section, like this:

define_function(<function identifier>)
{
  ...your code here
}

Functions can be overloaded as in C++ and other programming languages. You can declare two functions with the same name as long as they differ in the type or number of arguments. One example of overloaded functions can be found in the Hash module, it has two functions for calculating MD5 hashes, one receiving an offset and length within the file and another one receiving a string:

begin_declarations;

    declare_function("md5", "ii", "s", data_md5);
    declare_function("md5", "s", "s", string_md5);

end_declarations;

We are going to discuss function implementation more in depth in the More about functions section.

Accessing the scanned data¶

Most YARA modules needs to access the file or process memory being scanned to extract information from it. The data being scanned is sent to the module in the YR_SCAN_CONTEXT structure passed to the module_load function. The data is sometimes sliced in blocks, therefore your module needs to iterate over the blocks by using the foreach_memory_block macro:

int module_load(
    YR_SCAN_CONTEXT* context,
    YR_OBJECT* module_object,
    void* module_data,
    size_t module_data_size)
{
    YR_MEMORY_BLOCK* block;

    foreach_memory_block(context, block)
    {
        ..do something with the current memory block
    }
}

Each memory block is represented by a YR_MEMORY_BLOCK structure with the following attributes:

YR_MEMORY_BLOCK_FETCH_DATA_FUNC fetch_data¶

Pointer to a function returning a pointer to the block’s data.

size_t size¶

Size of the data block.

size_t base¶

Base offset/address for this block. If a file is being scanned this field contains the offset within the file where the block begins, if a process memory space is being scanned this contains the virtual address where the block begins.

The blocks are always iterated in the same order as they appear in the file or process memory. In the case of files the first block will contain the beginning of the file. Actually, a single block will contain the whole file’s content in most cases, but you can’t rely on that while writing your code. For very big files YARA could eventually split the file into two or more blocks, and your module should be prepared to handle that.

The story is very different for processes. While scanning a process memory space your module will definitely receive a large number of blocks, one for each committed memory region in the process address space.

However, there are some cases where you don’t actually need to iterate over the blocks. If your module just parses the header of some file format you can safely assume that the whole header is contained within the first block (put some checks in your code nevertheless). In those cases you can use the first_memory_block macro:

int module_load(
    YR_SCAN_CONTEXT* context,
    YR_OBJECT* module_object,
    void* module_data,
    size_t module_data_size)
{
    YR_MEMORY_BLOCK* block;
    uint8_t* block_data;

    block = first_memory_block(context);
    block_data = block->fetch_data(block)

    if (block_data != NULL)
    {
      ..do something with the memory block
    }
}

In the previous example you can also see how to use the fetch_data function. This function, which is a member of the YR_MEMORY_BLOCK structure, receives a pointer to the same block (as a self or this pointer) and returns a pointer to the block’s data. Your module doesn’t own the memory pointed to by this pointer, freeing that memory is not your responsibility. However keep in mind that the pointer is valid only until you ask for the next memory block. As long you use the pointer within the scope of a foreach_memory_block you are on the safe side. Also take into account that fetch_data can return a NULL pointer, your code must be prepared for that case.

uint8_t* block_data;

foreach_memory_block(context, block)
{
  block_data = block->fetch_data(block);

  if (block_data != NULL)
  {
    // using block_data is safe here.
  }
}

// the memory pointed to by block_data can be already freed here.

Setting variable’s values¶

The module_load function is where you assign values to the variables declared in the declarations section, once you’ve parsed or analized the scanned data and/or any additional module’s data. This is done by using the set_integer and set_string functions:

void set_integer(int64_t value, YR_OBJECT* object, const char* field, ...)¶

void set_string(const char* value, YR_OBJECT* object, const char* field, ...)¶

Both functions receive a value to be assigned to the variable, a pointer to a YR_OBJECT representing the variable itself or some ancestor of that variable, a field descriptor, and additional arguments as defined by the field descriptor.

If we are assigning the value to the variable represented by object itself, then the field descriptor must be NULL. For example, assuming that object points to a YR_OBJECT structure corresponding to some integer variable, we can set the value for that integer variable with:

set_integer(<value>, object, NULL);

The field descriptor is used when you want to assign the value to some descendant of object. For example, consider the following declarations:

begin_declarations;

    begin_struct("foo");

        declare_string("bar");

        begin_struct("baz");

            declare_integer("qux");

        end_struct("baz");

    end_struct("foo");

end_declarations;

If object points to the YR_OBJECT associated to the foo structure you can set the value for the bar string like this:

set_string(<value>, object, "bar");

And the value for qux like this:

set_integer(<value>, object, "baz.qux");

Do you remember that the module_object argument for module_load was a pointer to a YR_OBJECT? Do you remember that this YR_OBJECT is an structure just like bar is? Well, you could also set the values for bar and qux like this:

set_string(<value>, module_object, "foo.bar");
set_integer(<value>, module_object, "foo.baz.qux");

But what happens with arrays? How can I set the value for array items? If you have the following declarations:

begin_declarations;

    declare_integer_array("foo");

    begin_struct_array("bar")

        declare_string("baz");
        declare_integer_array("qux");

    end_struct_array("bar");

end_declarations;

Then the following statements are all valid:

set_integer(<value>, module, "foo[0]");
set_integer(<value>, module, "foo[%i]", 2);
set_string(<value>, module, "bar[%i].baz", 5);
set_string(<value>, module, "bar[0].qux[0]");
set_string(<value>, module, "bar[0].qux[%i]", 0);
set_string(<value>, module, "bar[%i].qux[%i]", 100, 200);

Those %i in the field descriptor are replaced by the additional integer arguments passed to the function. This work in the same way than printf in C programs, but the only format specifiers accepted are %i and %s, for integer and string arguments respectively.

The %s format specifiers is used for assigning values to a certain key in a dictionary:

set_integer(<value>, module, "foo[\"key\"]");
set_integer(<value>, module, "foo[%s]", "key");
set_string(<value>, module, "bar[%s].baz", "another_key");

If you don’t explicitely assign a value to a declared variable, array or dictionary item it will remain in undefined state. That’s not a problem at all, and is even useful in many cases. For example, if your module parses files from certain format and it receives one from a different format, you can safely leave all your variables undefined instead of assigning them bogus values that doesn’t make sense. YARA will handle undefined values in rule conditions as described in Using modules.

In addition to set_integer and set_string functions you have their get_integer and get_string counterparts. As the names suggest they are used for getting the value of a variable, which can be useful in the implementation of your functions to retrieve values previously stored by module_load.

int64_t get_integer(YR_OBJECT* object, const char* field, ...)¶

char* get_string(YR_OBJECT* object, const char* field, ...)¶

There’s also a function to the get any YR_OBJECT in the objects tree:

YR_OBJECT* get_object(YR_OBJECT* object, const char* field, ...)¶

Here goes a little exam...

Are the following two lines equivalent? Why?

set_integer(1, get_object(module_object, "foo.bar"), NULL);
set_integer(1, module_object, "foo.bar");

Storing data for later use¶

Sometimes the information stored directly in your variables by means of set_integer and set_string is not enough. You may need to store more complex data structures or information that don’t need to be exposed to YARA rules.

Storing information is essential when your module exports functions to be used in YARA rules. The implementation of these functions usually require to access information generated by module_load which must kept somewhere. You may be tempted to define global variables where to put the required information, but this would make your code non-thread-safe. The correct approach is using the data field of the YR_OBJECT structures.

Each YR_OBJECT has a void* data field which can be safely used by your code to store a pointer to any data you may need. A typical pattern is using the data field of the module’s YR_OBJECT, like in the following example:

typedef struct _MY_DATA
{
   int some_integer;

} MY_DATA;

int module_load(
    YR_SCAN_CONTEXT* context,
    YR_OBJECT* module_object,
    void* module_data,
    size_t module_data_size)
{
    module->data = yr_malloc(sizeof(MY_DATA));
    ((MY_DATA*) module_object->data)->some_integer = 0;

    return ERROR_SUCCESS;
}

Don’t forget to release the allocated memory in the module_unload function:

int module_unload(
    YR_OBJECT* module_object)
{
    yr_free(module_object->data);

    return ERROR_SUCCESS;
}

Function arguments¶

Within the function’s code you get its arguments by using integer_argument(n), float_argument(n), regexp_argument(n), string_argument(n) or sized_string_argument(n) depending on the type of the argument, where n is the 1-based argument’s number.

string_argument(n) can be used when your function expects to receive a NULL-terminated C string, if your function can receive arbitrary binary data possibly containing NULL characters you must use sized_string_argument(n).

Here you have some examples:

int64_t arg_1 = integer_argument(1);
RE_CODE arg_2 = regexp_argument(2);
char* arg_3 = string_argument(3);
SIZED_STRING* arg_4 = sized_string_argument(4);
double arg_5 = float_argument(1);

The C type for integer arguments is int64_t, for float arguments is double, for regular expressions is RE_CODE, for NULL-terminated strings is char* and for string possibly contaning NULL characters is SIZED_STRING*. SIZED_STRING structures have the following attributes:

SIZED_STRING¶

length¶

String’s length.

c_string¶

char* pointing to the string content.

Return values¶

Functions can return three types of values: strings, integers and floats. Instead of using the C return statement you must use return_string(x), return_integer(x) or return_float(x) to return from a function, depending on the function’s return type. In all cases x is a constant, variable, or expression evaluating to char*, int64_t or double respectively.

You can use return_string(UNDEFINED), return_float(UNDEFINED) and return_integer(UNDEFINED) to return undefined values from the function. This is useful in many situations, for example if the arguments passed to the functions don’t make sense, or if your module expects a particular file format and the scanned file is from another format, or in any other case where your function can’t a return a valid value.

Accessing objects¶

While writing a function we sometimes need to access values previously assigned to module’s variables, or additional data stored in the data field of YR_OBJECT structures as discussed earlier in Storing data for later use. But for that we need a way to get access to the corresponding YR_OBJECT first. There are two functions to do that: module() and parent(). The module() function returns a pointer to the top-level YR_OBJECT corresponding to the module, the same one passed to the module_load function. The parent() function returns a pointer to the YR_OBJECT corresponding to the structure where the function is contained. For example, consider the following code snipet:

define_function(f1)
{
    YR_OBJECT* module = module();
    YR_OBJECT* parent = parent();

    // parent == module;
}

define_function(f2)
{
    YR_OBJECT* module = module();
    YR_OBJECT* parent = parent();

    // parent != module;
}

begin_declarations;

    declare_function("f1", "i", "i", f1);

    begin_struct("foo");

        declare_function("f2", "i", "i", f2);

    end_struct("foo");

end_declarations;

In f1 the module variable points to the top-level YR_OBJECT as well as the parent variable, because the parent for f1 is the module itself. In f2 however the parent variable points to the YR_OBJECT corresponding to the foo structure while module points to the top-level YR_OBJECT as before.

Scan context¶

From within a function you can also access the YR_SCAN_CONTEXT structure discussed earlier in Accessing the scanned data. This is useful for functions which needs to inspect the file or process memory being scanned. This is how you get a pointer to the YR_SCAN_CONTEXT structure:

YR_SCAN_CONTEXT* context = scan_context();

Data structures¶

YR_COMPILER¶

Data structure representing a YARA compiler.

YR_MATCH¶

Data structure representing a string match.

int64_t base¶

Base offset/address for the match. While scanning a file this field is usually zero, while scanning a process memory space this field is the virtual address of the memory block where the match was found.

int64_t offset¶

Offset of the match relative to base.

int32_t match_length¶

Length of the matching string

uint8_t* data¶

Pointer to a buffer containing a portion of the matching string.

int32_t data_length¶

Length of data buffer. data_length is the minimum of match_length and MAX_MATCH_DATA.

Changed in version 3.5.0.

YR_META¶

Data structure representing a metadata value.

const char* identifier¶

Meta identifier.

int32_t type¶

One of the following metadata types:

META_TYPE_NULL META_TYPE_INTEGER META_TYPE_STRING META_TYPE_BOOLEAN

YR_MODULE_IMPORT¶

const char* module_name¶

Name of the module being imported.

void* module_data¶

Pointer to additional data passed to the module. Initially set to NULL, your program is responsible of setting this pointer while handling the CALLBACK_MSG_IMPORT_MODULE message.

size_t module_data_size¶

Size of additional data passed to module. Your program must set the appropriate value if module_data is modified.

YR_RULE¶

Data structure representing a single rule.

const char* identifier¶

Rule identifier.

const char* tags¶

Pointer to a sequence of null terminated strings with tag names. An additional null character marks the end of the sequence. Example: tag1\0tag2\0tag3\0\0. To iterate over the tags you can use yr_rule_tags_foreach().

YR_META* metas¶

Pointer to a sequence of YR_META structures. To iterate over the structures use yr_rule_metas_foreach().

YR_STRING* strings¶

Pointer to a sequence of YR_STRING structures. To iterate over the structures use yr_rule_strings_foreach().

YR_RULES¶

Data structure representing a set of compiled rules.

YR_STREAM¶

New in version 3.4.0.

Data structure representing a stream used with functions yr_rules_load_stream() and yr_rules_save_stream().

void* user_data¶

A user-defined pointer.

YR_STREAM_READ_FUNC read¶

A pointer to the stream’s read function provided by the user.

YR_STREAM_WRITE_FUNC write¶

A pointer to the stream’s write function provided by the user.

YR_STRING¶

Data structure representing a string declared in a rule.

const char* identifier¶

String identifier.

Functions¶

int yr_initialize(void)¶

Initalize the library. Must be called by the main thread before using any other function. Return ERROR_SUCCESS on success another error code in case of error. The list of possible return codes vary according to the modules compiled into YARA.

int yr_finalize(void)¶

Finalize the library. Must be called by the main free to release any resource allocated by the library. Return ERROR_SUCCESS on success another error code in case of error. The list of possible return codes vary according to the modules compiled into YARA.

void yr_finalize_thread(void)¶

Any thread using the library, except the main thread, must call this function when it finishes using the library.

int yr_compiler_create(YR_COMPILER** compiler)¶

Create a YARA compiler. You must pass the address of a pointer to a YR_COMPILER, the function will set the pointer to the newly allocated compiler. Returns one of the following error codes:

ERROR_SUCCESS

ERROR_INSUFICENT_MEMORY

void yr_compiler_destroy(YR_COMPILER* compiler)¶

Destroy a YARA compiler.

void yr_compiler_set_callback(YR_COMPILER* compiler, YR_COMPILER_CALLBACK_FUNC callback, void* user_data)¶

Changed in version 3.3.0.

Set a callback for receiving error and warning information. The user_data pointer is passed to the callback function.

int yr_compiler_add_file(YR_COMPILER* compiler, FILE* file, const char* namespace, const char* file_name)¶

Compile rules from a file. Rules are put into the specified namespace, if namespace is NULL they will be put into the default namespace. file_name is the name of the file for error reporting purposes and can be set to NULL. Returns the number of errors found during compilation.

int yr_compiler_add_string(YR_COMPILER* compiler, const char* string, const char* namespace_)¶

Compile rules from a string. Rules are put into the specified namespace, if namespace is NULL they will be put into the default namespace. Returns the number of errors found during compilation.

int yr_compiler_get_rules(YR_COMPILER* compiler, YR_RULES** rules)¶

Get the compiled rules from the compiler. Returns one of the following error codes:

ERROR_SUCCESS

ERROR_INSUFICENT_MEMORY

int yr_compiler_define_integer_variable(YR_COMPILER* compiler, const char* identifier, int64_t value)¶

Defines an integer external variable.

int yr_compiler_define_float_variable(YR_COMPILER* compiler, const char* identifier, double value)¶

Defines a float external variable.

int yr_compiler_define_boolean_variable(YR_COMPILER* compiler, const char* identifier, int value)¶

Defines a boolean external variable.

int yr_compiler_define_string_variable(YR_COMPILER* compiler, const char* identifier, const char* value)¶

Defines a string external variable.

void yr_rules_destroy(YR_RULES* rules)¶

Destroy compiled rules.

int yr_rules_save(YR_RULES* rules, const char* filename)¶

Save rules into the file specified by filename. Returns one of the following error codes:

ERROR_SUCCESS

ERROR_COULD_NOT_OPEN_FILE

int yr_rules_save_stream(YR_RULES* rules, YR_STREAM* stream)¶

New in version 3.4.0.

Save rules into stream. Returns one of the following error codes:

ERROR_SUCCESS

int yr_rules_load(const char* filename, YR_RULES** rules)¶

Load rules from the file specified by filename. Returns one of the following error codes:

ERROR_SUCCESS

ERROR_INSUFICENT_MEMORY

ERROR_COULD_NOT_OPEN_FILE

ERROR_INVALID_FILE

ERROR_CORRUPT_FILE

ERROR_UNSUPPORTED_FILE_VERSION

int yr_rules_load_stream(YR_STREAM* stream, YR_RULES** rules)¶

New in version 3.4.0.

Load rules from stream. Returns one of the following error codes:

ERROR_SUCCESS

ERROR_INSUFICENT_MEMORY

ERROR_INVALID_FILE

ERROR_CORRUPT_FILE

ERROR_UNSUPPORTED_FILE_VERSION

int yr_rules_scan_mem(YR_RULES* rules, uint8_t* buffer, size_t buffer_size, int flags, YR_CALLBACK_FUNC callback, void* user_data, int timeout)¶

Scan a memory buffer. Returns one of the following error codes:

ERROR_SUCCESS

ERROR_INSUFICENT_MEMORY

ERROR_TOO_MANY_SCAN_THREADS

ERROR_SCAN_TIMEOUT

ERROR_CALLBACK_ERROR

ERROR_TOO_MANY_MATCHES

int yr_rules_scan_file(YR_RULES* rules, const char* filename, int flags, YR_CALLBACK_FUNC callback, void* user_data, int timeout)¶

Scan a file. Returns one of the following error codes:

ERROR_SUCCESS

ERROR_INSUFICENT_MEMORY

ERROR_COULD_NOT_MAP_FILE

ERROR_ZERO_LENGTH_FILE

ERROR_TOO_MANY_SCAN_THREADS

ERROR_SCAN_TIMEOUT

ERROR_CALLBACK_ERROR

ERROR_TOO_MANY_MATCHES

int yr_rules_scan_fd(YR_RULES* rules, YR_FILE_DESCRIPTOR fd, int flags, YR_CALLBACK_FUNC callback, void* user_data, int timeout)¶

Scan a file descriptor. In POSIX systems YR_FILE_DESCRIPTOR is an int, as returned by the open() function. In Windows YR_FILE_DESCRIPTOR is a HANDLE as returned by CreateFile().

Returns one of the following error codes:

ERROR_SUCCESS

ERROR_INSUFICENT_MEMORY

ERROR_COULD_NOT_MAP_FILE

ERROR_ZERO_LENGTH_FILE

ERROR_TOO_MANY_SCAN_THREADS

ERROR_SCAN_TIMEOUT

ERROR_CALLBACK_ERROR

ERROR_TOO_MANY_MATCHES

yr_rule_tags_foreach(rule, tag)¶

Iterate over the tags of a given rule running the block of code that follows each time with a different value for tag of type const char*. Example:

const char* tag;

/* rule is a YR_RULE object */

yr_rule_tags_foreach(rule, tag)
{
  ..do something with tag
}

yr_rule_metas_foreach(rule, meta)¶

Iterate over the YR_META structures associated to a given rule running the block of code that follows each time with a different value for meta. Example:

YR_META* meta;

/* rule is a YR_RULE object */

yr_rule_metas_foreach(rule, meta)
{
  ..do something with meta
}

yr_rule_strings_foreach(rule, string)¶

Iterate over the YR_STRING structures associated to a given rule running the block of code that follows each time with a different value for string. Example:

YR_STRING* string;

/* rule is a YR_RULE object */

yr_rule_strings_foreach(rule, string)
{
  ..do something with string
}

yr_string_matches_foreach(string, match)¶

Iterate over the YR_MATCH structures associated to a given string running the block of code that follows each time with a different value for match. Example:

YR_MATCH* match;

/* string is a YR_STRING object */

yr_string_matches_foreach(string, match)
{
  ..do something with match
}

yr_rules_foreach(rules, rule)¶

Iterate over each YR_RULE in a YR_RULES object running the block of code that follows each time with a different value for rule. Example:

YR_RULE* rule;

/* rules is a YR_RULES object */

yr_rules_foreach(rules, rule)
{
  ..do something with rule
}

Error codes¶

ERROR_SUCCESS¶

Everything went fine.

ERROR_INSUFICENT_MEMORY¶

Insuficient memory to complete the operation.

ERROR_COULD_NOT_OPEN_FILE¶

File could not be opened.

ERROR_COULD_NOT_MAP_FILE¶

File could not be mapped into memory.

ERROR_ZERO_LENGTH_FILE¶

File length is zero.

ERROR_INVALID_FILE¶

File is not a valid rules file.

ERROR_CORRUPT_FILE¶

Rules file is corrupt.

ERROR_UNSUPPORTED_FILE_VERSION¶

File was generated by a different YARA and can’t be loaded by this version.

ERROR_TOO_MANY_SCAN_THREADS¶

Too many threads trying to use the same YR_RULES object simultaneosly. The limit is defined by MAX_THREADS in ./include/yara/limits.h

ERROR_SCAN_TIMEOUT¶

Scan timed out.

ERROR_CALLBACK_ERROR¶

Callback returned an error.

ERROR_TOO_MANY_MATCHES¶

Too many matches for some string in your rules. This usually happens when your rules contains very short or very common strings like 01 02 or FF FF FF FF. The limit is defined by MAX_STRING_MATCHES in ./include/yara/limits.h