Monday, June 18, 2007

Escaping comma and space in GNU Make

Sometimes you need to hide a comma or a space from GNU Make's parser because GNU Make might strip it (if it's a space) or interpret it as an argument separator (for example, in a function invocation).

First the problem. If you wanted to change every , into a ; in a string in GNU Make you'd probably head for the $(subst) function and do the following:

$(subst ,,;,$(string))

See the problem? The argument separator for functions in GNU Make is , and hence the first , (the search text) is considered to be separator. Hence the search text in the above is actually the empty string, the replacement text is also the empty string and the ;, is just preprended to whatever is in $(string).

A similar problem occurs with spaces. Suppose you want to replace all spaces with ; in a string. You get a similar problem with $(subst), this time because the leading space is stripped:

$(subst ,;,$(string))

That extra space isn't an argument it's just extraneous whitespace and hence it is ignored. GNU Make just ends up appending ; to the $(string).

So, how do you solve this?

The answer is define variables that contain just a comma and just a space and use them. Because the argument splitting is done before variable expansion it's possible to have an argument that's a comma or a space.

For a comma you just do:

comma := ,
$(subst $(comma),;,$(string))

And everything works.

For a space you need to get a space into a string, I find the easiest way is like this:

space :=
space +=
$(subst $(space),;,$(string))

That works because += always space separates the value of the variable with the appended text.

Now, GNU Make has really liberal variable naming rules. Pretty much anything goes, so it's possible to define a variable with the name , or even having the name consisting of a space character.

First, here's how to define them:

, := ,
space :=
space +=
$(space) :=
$(space) +=

The first line is clear, it does an immediate define of a , to the variable named ,. The second one is a little more complex. First, I define a variable called space which contains a space character and then I use it to define a variable whose name is that space character.

You can verify that these work using $(warning) (I like to wrap the variable being printed in square brackets for absolute certainty of the content):

$(warning [$(,)])
$(warning [$( )])

$ make
Makefile:1: [,]
Makefile:2: [ ]

Yes, that's pretty odd looking, but it gets stranger. Since GNU Make will interpret $ followed by a single character as a variable expansion you can drop the braces and write:

$(warning [$,])
$(warning [$ ])

Now that starts to look like escaping. In the examples above you can use these variables to make things a little clearer:

$(subst $(,),;,$(string))
$(subst $ ,;,$(string))

Note that you have to use the $(,) form because function argument splitting occurs before the expansion and GNU Make gets confused. In the second line the space is 'escaped' with the $ sign.

You might be wondering about other crazy variable names: here are a few that's possible with GNU Make:

# Defining the $= or $(=) variable which has the value =
equals := =
$(equals) := =

# Define the $# or $(#) variable which has the value #
hash := \#
$(hash) := \#

# Define the $: or $(:) variable which has the value :
colon := :
$(colon) := :

# Define the $($$) variable which has the value $
dollar := $$
$(dollar) := $$

; := ;
% := %

You probably don't need any of those, but you never know...


madscientist said...

I don't recommend using a space as a variable name. I've toyed with the idea of allowing some special syntactic sugar for user-defined functions, that would allow you to omit the $(call ...) prefix and just invoke the user-defined function directly, like:

$(my-func arg1, arg2)

Obviously the way this works is that if make detects a space in the variable name, it would assume that it is a user-defined function. If this were to happen I'd probaby make whitespace in variable names illegal, just to cut down on confusion and bizarre misbehaviors.

serious said...

@John Graham-Cumming

Thanks a lot guy, it works perfectly well.

G Mack said...

After actually trying this I have discovered that your space function doesn't do what you think it does and my best guess is that even though it looks like a space it's actually EOL.

The Make help page suggests defining a space this way:
space:= $(empty) $(empty)

Unknown said...

Thanks for the info. How can I escape a newline in gnu make so I can do

$(eval $(NEWLINE)ifndef X$(NEWLINE)X=Y$(NEWLINE)endif)



Unknown said...

Thanks for the info. How can I escape a newline in gnu make so I can do

$(eval $(NEWLINE)ifndef X$(NEWLINE)X=Y$(NEWLINE)endif)



Unknown said...

define NEWLINE


seems to define a NEWLINE in whitespace, so I'm set! Thanks.

C.Alberto said...

Thank you very much,

finally I was able to embed perl programs into just one gmake program.

It is extremely useful when you have to parse&mangle automatically generated files before compilation

C.Alberto said...

Thank you very much.

Your post is precious to use embedded perl in GMAKE programs.

Thus now it is possible to parse and process automatically generated programs.

Cube said...

Thanks for these crazy tips. I was trying out some of them and looking at how the Make database stores them (--print-data-base).

There are two ways that I prefer for getting the $( ) space variable set in one line:

$() $() := $() $()

which is essentially the $(empty) solution but using $() with the assumption that it always resolves to nil, and

$(subst ,, ) := $(subst ,, )

$(subst ,, ) is how the Make database appears to be storing a space as assigned by any of the above methods.