Makefile::Parser::GmakeDB(3) GNU makefile parser using GNU make's database dump

VERSION

This document describes Makefile::Parser::GmakeDB 0.215 released on 18 August 2011.

SYNOPSIS


use Makefile::Parser::GmakeDB;
my $db_listing = `make --print-data-base -pqRrs -f Makefile`;
my $ast = Makefile::Parser::GmakeDB->parse(\$db_listing);

DESCRIPTION

This module serves as a parser for GNU makefiles. However, it does not parse user's original makefile directly. Instead it uses Makefile::DOM to parse the ``data base output listing'' produced by GNU make (via its "--print-data-base" option). So essentially it reuses the C implementation of GNU make.

This parser has been tested as a component of the pgmake-db utility and has successfully passed 51% of GNU make 3.81's official test suite.

The result of the parser is a makefile AST defined by Makefile::AST.

The ``data base output listing'' generated by "make --print-data-base" is a detailed listing for GNU make's internal data structures, which is essentially the AST used by "make". According to GNU make's current maintainer, Paul Smith, this feature is provided primarily for debugging the user's own makefiles, and it also helps the GNU make developer team to diagnose the flaws in make itself. Incidentally this output is conformed to the GNU makefile syntax, and a lot of important information is provided in the form of makefile comments. Therefore, my GmakeDB parser is able to reuse the Makefile::DOM module to parse this output listing.

The data base output from GNU make can be divided into several clearly-separated segments. They're file header, ``Variables'', ``Files'', ``VPATH Search Paths'', as well as the last resource stats information.

The contents of these segments are mostly obvious. The Files segment may deserve some explanation. It is the place for explicit rules.

Now let's take the Variables segment as an example to demonstrate the format of the data base listing: 

    # Variables
    # automatic
    <D = $(patsubst %/,%,$(dir $<))
    # automatic
    ?F = $(notdir $?)
    # environment
    DESKTOP_SESSION = default
    # automatic
    ?D = $(patsubst %/,%,$(dir $?))
    # environment
    GTK_RC_FILES = /etc/gtk/gtkrc:/home/agentz/.gtkrc-1.2-gnome2
    # environment
    ...

It's shown that the flavor and origin of the makefile variables are given in the previous line as comments. Hence feeding this back into GNU make again makes little sense.

Similarly, the Files segment for explicit rules also puts big amount of the important information into makefile comments: 

    # Files
    # Not a target:
    bar.c:
    #  Implicit rule search has not been done.
    #  Modification time never checked.
    #  File has not been updated.
    all: foo.o bar.o
    #  Implicit rule search has been done.
    #  File does not exist.
    #  File has not been updated.
    # variable set hash-table stats:
    # Load=0/32=0%, Rehash=0, Collisions=0/0=0%
    foo.o: foo.c
    #  Implicit rule search has not been done.
    #  Implicit/static pattern stem: `foo'
    #  File does not exist.
    #  File has not been updated.
    # variable set hash-table stats:
    # Load=0/32=0%, Rehash=0, Collisions=0/0=0%
    #  commands to execute (from `ex2.mk', line 8):
        $(CC) -c $(CFLAGS) $< -o $@
    ...

From the previous two data base listing snippets, it's not hard to see that the variable references in rule commands and recursively-expanded variables's values are not expanded.

Experiments have shown that GNU make will do implicit rule search for the first rule that needs to, but no more. This behavior means testing our own implicit rule searching algorithm requires specifying at least two goals that require matching.

DEPENDENCIES

GNU make 3.81
At least the make executable of GNU make 3.81 is required to work with this module.
Makefile::DOM

BUGS

  • GNU make does not escape meta characters appeared in rule targets and prerequisites in its data base listing. Examples are ":", "\", and "#". This bug has been reported to the GNU make team as "Savannah bug #20067".

    This bug has not yet been fixed on the "make" side, so I have to work around this issue by preprocessing the data base listing in the makesimple script.

  • The data base listing produced by GNU make lacks the information regarding the "export" and "unexport" directives. It gives rise to the lack of information in the resulting AST structures constructed by this module. Hence the current AST and runtime do not implement the "export" and "unexport" directives.

    To make it even worse, there's no known way to work around it.

    I've already reported this issue to the GNU make team as Savannah bug #20069.

CODE REPOSITORY

For the very latest version of this script, check out the source from

http://github.com/agentzh/makefile-parser-pm <http://github.com/agentzh/makefile-parser-pm>.

There is anonymous access to all.

AUTHOR

Zhang ``agentzh'' Yichun "<[email protected]>"

COPYRIGHT AND LICENSE

Copyright (c) 2005-2008 by Zhang ``agentzh'' Yichun (agentzh).

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

LAST SEARCHED