Unikado.PHPLibs.Tokenizer is a .NET library for tokenize some PHP code, without the use of PHP it self.

Its faster then PHPs tokenizer and it gives you a lot more detailed token informations.

But its not a ready to use application! Its a library (*.dll), usable for .NET programmers, to create own PHP helping applications from.

Author: Ulf -UniKado- Kadner
License: LGPL (Lesser GNU Public License)
Requires: MS .NET Framework v4.5 or higher
Requires: MS Visual Studio 2013 or higher

The source code is a Visual Studio 2013 Project. (see requirements) There is no really need for VS2013 but its the easiest way. For lower versions of Visual Studio you must convert the project or copy contents into your working project. There are no special depencies. The code will work with .NET v3.5 or higher.


// We are inside a class here… :-P 

// The private field for holding the current tokens. 
private Unikado.PHPLibs.Tokenizer.TokenCollection tokens;

private bool getTokens()
    // define the PHP file path 
    string phpFile = @"C:\Users\Aurora\Documents\PHP-Projects\example\foo.php";
    // Specify the encoding of the PHP file 
    var encoding = new UTF8Encoding(false);
    // You have to catch alle parse errors 
        // Getting all tokens (Its a TokenCollection)
        tokens = Unikado.PHPLibs.Tokenizer.PhpTokenizer.ParseFile(phpFile, encoding);
    catch (Exception ex)
        return false;
    return tokens.Count > 0;

Last edited May 2, 2014 at 2:42 PM by Unikado, version 5