Vigdís — LFE Friday, LiffyBot studying Erlang modules in a Mead-style command room

LFE Friday - string:tokens/2

Heads up: This post was written for LFE 0.9 / Erlang/OTP 17. The current release is LFE 2.2.0. APIs and examples may be outdated.

This week's LFE Friday was translated with permission from the Erlang Thursday series by Steven Proctor. This week's translator: Robert Virding.

Today’s LFE Friday is string:tokens/2.

string:tokens/2 takes a string as the first argument, and a list of separators to split the string on, and returns a list of token strings.

> (string:tokens "foo" "")
("foo")
> (string:tokens "banana" "a")
("b" "n" "n")
> (string:tokens "It was the best of times, it was the worst of times" " ") 
("It" "was" "the" "best" "of" "times," "it" "was" "the" "worst" "of" "times")

If consecutive separators appear in the string they will be treated as a single separator, and no empty strings will be returned.

> (string:tokens "Mississippi" "s")
("Mi" "i" "ippi")
> (string:tokens "Mississippi" "sp")
("Mi" "i" "i" "i")
> (string:tokens "Mississippi" "is")
("M" "pp")

The order of the separators in the separator list passed to string:tokens/2 does not matter, and can be specified in any order.

> (string:tokens "Mississippi" "ps")                                       
("Mi" "i" "i" "i")
> (string:tokens "Mississippi" "sp")
("Mi" "i" "i" "i")

And as the separator list is just simply a list of separators, instead of passing a string, the integer values for the characters to use as the separators can be passed as a list, as a list of the integers is the same as a string.

> #\s
115
> #\p
112
> '(115 112)
"sp"
> (string:tokens "Mississippi" '(115))
("Mi" "i" "ippi")
> (string:tokens "Mississippi" '(115 112))
("Mi" "i" "i" "i")

–Proctor, Robert