strutil::Tokenizer Class Reference

String Tokenizer. More...

#include <strutil.h>

Collaboration diagram for strutil::Tokenizer:

List of all members.

Public Member Functions
	Tokenizer (const std::string &str)
	Tokenizer (const std::string &str, const std::string &delimiters)
bool	nextToken ()
bool	nextToken (const std::string &delimiters)
const std::string	getToken () const
void	reset ()
Static Public Attributes
static const std::string	DEFAULT_DELIMITERS
Protected Attributes
size_t	m_Offset
const std::string	m_String
std::string	m_Token
std::string	m_Delimiters

---

Detailed Description

String Tokenizer.

String tokenizer, which separate the whole string to tokens.

Definition at line 56 of file strutil.h.

---

Constructor & Destructor Documentation

strutil::Tokenizer::Tokenizer

(

const std::string &

str

)

Definition at line 105 of file StrUtil.cpp.

        : m_String(str), m_Offset(0), m_Delimiters(DEFAULT_DELIMITERS) {}

strutil::Tokenizer::Tokenizer	(	const std::string &	str,
		const std::string &	delimiters
	)

Definition at line 108 of file StrUtil.cpp.

        : m_String(str), m_Offset(0), m_Delimiters(delimiters) {}

---

Member Function Documentation

const string strutil::Tokenizer::getToken

(

)

const

Definition at line 137 of file StrUtil.cpp.

                                           {
        return m_Token;
    }

Here is the caller graph for this function:

bool strutil::Tokenizer::nextToken

(

const std::string &

delimiters

)

Definition at line 115 of file StrUtil.cpp.

                                                         {
        // find the start charater of the next token.
        size_t i = m_String.find_first_not_of(delimiters, m_Offset);
        if (i == string::npos) {
            m_Offset = m_String.length();
            return false;
        }

        // find the end of the token.
        size_t j = m_String.find_first_of(delimiters, i);
        if (j == string::npos) {
            m_Token = m_String.substr(i);
            m_Offset = m_String.length();
            return true;
        }

        // to intercept the token and save current position
        m_Token = m_String.substr(i, j - i);
        m_Offset = j;
        return true;
    }

bool strutil::Tokenizer::nextToken

(

)

Definition at line 111 of file StrUtil.cpp.

                              {
        return nextToken(m_Delimiters);
    }

Here is the caller graph for this function:

void strutil::Tokenizer::reset

(

)

to reset the tokenizer. After reset it, the tokenizer can get the tokens from the first token.

Definition at line 141 of file StrUtil.cpp.

                          {
        m_Offset = 0;
    }

---

Member Data Documentation

const string strutil::Tokenizer::DEFAULT_DELIMITERS [static]

Definition at line 59 of file strutil.h.

std::string strutil::Tokenizer::m_Delimiters [protected]

Definition at line 77 of file strutil.h.

size_t strutil::Tokenizer::m_Offset [protected]

Definition at line 74 of file strutil.h.

const std::string strutil::Tokenizer::m_String [protected]

Definition at line 75 of file strutil.h.

std::string strutil::Tokenizer::m_Token [protected]

Definition at line 76 of file strutil.h.

---

The documentation for this class was generated from the following files:

• core/Parser/strutil.h
• core/Parser/StrUtil.cpp