Split a string and keep the delimiter, a Tokenizer

Sometimes there isn’t a ready class or method to use in the .NET framework and you actually have to do some programming yourself. Alternatively you search and ask questions if it is done before.
 
There was a question on the MSDN forums asking how to return the delimiter with the String.Split result.
 
I ended up writing a small sample to do it and I thought I’d share it here. It is basic and just demonstrates the general idea how it can be implemented. It is compiling and running!

Tokenizer.cs (1.77 kb)

Leave a Reply

Your email address will not be published. Required fields are marked *