Bashs Brace Expansion is one of my favorite syntaxes of all time between any programming or scripting languages I have ever written in.
Its a fast way to produce a list as input, which they are in certain order or combination. Its very helpful and clear to be used in shell prompt. For example, from Gentoo Handbook:
cdimage ~# umount -l /mnt/gentoo/dev{/shm,/pts,}
This method enables user to type less but have same desired inputs. Compact and elegant. From time to time, I have even seen people writing in Brace Expansion to express things, and they are not necessary about Bash or even coding.
Contents
1 Syntax
The syntax of Brace Expansion isnt complicated, basically its just some expandable expressions wrapped by braces. Its very simple to read and quite understandable even you first see it. I recommend that you read Brace Expansion in Bash manual page first, try to read the descriptive explanation first, then continue to read here.
Although its simple, but I think using EBNF to explain may be a better idea and more thorough. This is my first time to write EBNF and I only have read one page. If you see any problem with it, please feel free to correct me.
Also keep in mind that I use my own terminology to explain, they may read different them in Bash manual page.
1.1 EBNF
brace = [ preamble ], expandable, [ postscript ] ; preamble = postscript = string or brace = string | brace ; expandable = "{", ( sequence | list ), "}" ; sequence = start, "..", end, [ "..", step ] ; start = end = number | single alphabet ; step = number ; number = [ "-" ], { "0" }, { digit } ; list = string or brace, ",", string or brace , { ",", string or brace } ;
The first line defines the structure of Brace Expansion. Its constituted of three parts:
- Preamble,
- Expandable, and
- Postscript.
The following sections will focus on the Expandable part first, which is either Sequence or List expressions.
1.2 Sequence
Sequence expression is used to quickly generate number sequence or alphabet sequence. Its form is as follows:
sequence = start, "..", end, [ "..", step ] ; start = end = number | single alphabet ; step = number ; number = [ "-" ], { "0" }, { digit } ;
It has two required parameters, start and end, and one optional parameter step which is the increment. If you already have programming language, you may have already been able to guess how to write a sequence.
To read the expression, I would suggest
Generate a sequence, starting from start and printing out every step element, until reach end.
1.2.1 Number sequence
To produce a number sequence, simply assign the starting number and end number. Here are some examples:
echo "{1..5} =" {1..5} echo "{5..1} =" {5..1} echo "{1..5..1} =" {1..5..1} echo "{5..1..-1} =" {5..1..-1}
{1..5} = 1 2 3 4 5 {5..1} = 5 4 3 2 1 {1..5..1} = 1 2 3 4 5 {5..1..-1} = 5 4 3 2 1
You can see two sets, the first sets step is omitted, but not the second set. Bash knows how to walk your sequence even you do not supply the step.
step is used to skip some numbers, for example, you print every two numbers from 1 to 5 and 5 down to 1:
echo "{1..5..2} =" {1..5..2} echo "{5..1..-2} =" {5..1..-2} echo echo "{5..1..2} =" {5..1..-2}
{1..5..2} = 1 3 5 {5..1..-2} = 5 3 1 {5..1..2} = 5 3 1
The first two results are what we expected. But the last one is interesting, even we set the step to negative number for a sequence in descending order, Bash still knows how to walk it correctly. In other words, you should set step always to be positive number, there is no need for you to change steps sign.
1.2.1.1 Zero-padding
The number sequence has a nice feature when you needs the numbers to be padded with zeros. You can either pad the start or end to indicate how many digits you need in the results, for example:
echo "{001..3} =" {001..3} echo "{1..003} =" {1..003}
{001..3} = 001 002 003 {1..003} = 001 002 003
Although its great, but you may have a problem with sequences crossing 0. For instance:
echo "{-1..003} =" {-1..003} echo "{-001..3} =" {-001..3}
{-1..003} = -01 000 001 002 003 {-001..3} = -001 0000 0001 0002 0003
The length of a number in character counts the negative sign in.
1.2.2 Alphabet sequence
The amazing part of Sequence is not only you can have numbers but also the letters. Its the only one language I know that you can generate a string of a-z in only 6 characters: {a..z}. When start and end is the type of alphabet, namely a-z or A-Z, the sequence would become a list of alphabets in similar fashion as number sequence. For example,
echo "{a..e} =" {a..e} echo "{e..a} =" {e..a} echo "{a..e..2} =" {a..e..2} echo "{e..a..2} =" {e..a..2}
{a..e} = a b c d e {e..a} = e d c b a {a..e..2} = a c e {e..a..2} = e c a
But dont use mixed cases like {X..c}, you will get unexpected results.
echo "{X..c} =" {X..c}
{X..c} = X Y Z [ ] ^ _ ` a b c
Even it prints out some non-alphabet characters, you still cant set characters other than alphabets as start and end.
1.3 List
List expression is a comma-separated of string and/or Brace Expansion, it has to have at least two items to be parsed as a List:
list = string or brace, ",", string or brace , { ",", string or brace } ;
The string is the normal Bash string, it can be unquoted or quoted with single-or-double quotation marksand it can also be an empty string ''. For example,
echo "{abc,\"def\",'123 456'} =" {abc,"def",'123 456'} echo "{{123,}} =" {{123,}}
{abc,"def",'123 456'} = abc def 123 456 {{123,}} = {123} {}
When an item is a Brace Expansion, its also a form of nesting. The following example shows that you can mix with strings and Brace Expansions.
echo "{abc,{\"def\",'123 456'}} =" {abc,{"def",'123 456'}}
{abc,{"def",'123 456'}} = abc def 123 456
1.4 Nesting
1.4.1 Items
In the end of List, an nesting example is shown. If you look at it and ask how many items of final result?
{abc,{"def",'123 456'}}
The answer is 3. The following sample code using array to demonstrate:
arr=({abc,{"def",'123 456'}}) echo ${arr[0]} echo ${arr[1]} echo ${arr[2]}
abc def 123 456
You can see it as a flattened list.
1.4.2 Multiple number sequence
The Sequence expression can only represent one sequence at a time, but what if you need 1 to 3 and 99 to 101? You can use separate Brace Expansions, for instance:
echo {1..3} {99..101}
1 2 3 99 100 101
But this actually have a slight disadvantage when Preamble and Postscript is needed. You will have to add them to two sequences. If you nest two sequences inside a Brace Expansion, it reads more clear:
echo foo{{1..3},{99..101}}bar
foo1bar foo2bar foo3bar foo99bar foo100bar foo101bar
2 Expanding
When expanding, Bash tries to expand Brace Expansion first, then other expansions.
2.1 Preamble and Postscript
brace = [ preamble ], expandable, [ postscript ] ;
When a Brace Expansion expands, each item of Expandable will be prefixed and appended with preamble and postscript, respectively. If preamble and/or postscript is actually a Brace Expansion, then a combination is performed between the items of preamble, exandable, and/or postscript.
A quick and simple example:
echo {1..3}{a,b,c}
1a 1b 1c 2a 2b 2c 3a 3b 3c
Note that the combination is performed before Expansions.
2.2 Expansions
After Brace Expansion is done, the other expansions will be expanded next.
ac=foo bc=bar echo {$a,$b,$(echo blah),}c a1=foo a2=bar echo $a{1..2}
foo bar blahc c foo bar
2.2.1 Tilde and Pathname Expansions
When Brace Expansion is expanded and Tilde and/or Pathname Expansions is formed in the result, they will also be expanded. If there is no match filename or ~user, then the string will be kept untouched, that is * or ? will not be removed. But if matches, then the item is replaced by the match item(s).
touch /tmp/ABC{,D} echo {/tmp/{AB*,NOSUCHFILE?},~{,nosuchuser}}
/tmp/ABC /tmp/ABCD /tmp/NOSUCHFILE? /home/livibetter ~nosuchuser
As you can see, first item is replaced by the filenames in /tmp/. The second item is left untouched, third item expands into home directory of current user, forth or last item is left untouched.
3 Compatibility
Brace Expansion is not a POSIX-compliant syntax. If you want to develop a POSIX shell script, you should consider to run Bash with +B or set +B in the beginning of script. When its set, the Brace Expansion is just like a normal string.
4 Conclusion
I hope this blog post have convinced you the amazing of Brace Expansion and would encourage to use it wherever it fits. However, with my personal experience, I hardly use it in scripting but only in shell prompt most of time. Like creating a quick backup or fixing a typo in filename:
cp somefile.ext{,.bak} touch some-serious-lng-filename.ext mv some-seriously-l{,o}ng-filename.ext
You may think number sequence would be very helpful in scripting, unfortunately thats only usable with pre-determined sequence. Yo cant do it like:
for {1..${#arr[]}}; do : done
That is not a valid sequence expression, therefore its seen as a literal {1..100} if the number of items is 100. However this is actually a good example or excuse why Parameter Expansion doesnt work with Sequence or why Brace expands before others. Imagine that there is no items in the array, what should happen next? Should Bash gives you a 1 0 sequence, or throw out an error?
I also want to point out, although you can use eval as workaround for parameter in sequence expression, but like any languages, its always not recommended and seen as taboo when doing something with such method. Think about this
a=1 b='$(echo This may be just echo)' eval echo {$a..$b}
{1..This may be just echo}
$b can be a number from a input file, this can be dangerous.
Anyway, its still very useful in shell prompt. Especially when you need to expand into some paths like this from LFS:
mkdir -pv /{bin,boot,etc/{opt,sysconfig},home,lib,mnt,opt,run}
Generally, Brace Expansion is a syntax for a quick generation of a list and you definitely should learn to use it.