gimbal_scanner.h File Reference

Go to the source code of this file.

Data Structures
struct	GblScannerCursor
struct	GblScannerClass
struct	GblScanner

Macros
#define	GBL_SCANNER_DELIMETERS_DEFAULT
Type System
Type UUID and cast operators
#define	GBL_SCANNER_TYPE
#define	GBL_SCANNER(self)
#define	GBL_SCANNER_CLASS(klass)
#define	GBL_SCANNER_GET_CLASS(self)
Scanning
Methods for scanning the next token
Peeking
Methods for peeking at the next token

Typedefs
typedef GblEnum	GBL_SCANNER_FLAGS

Enumerations
enum	GBL_SCANNER_FLAGS

Functions
Lifetime Management
Methods for managing GblScanner creation/destruction
GblScanner *	GblScanner_create (const char *pStr, size_t len)
GblScanner *	GblScanner_ref (GblScanner *pSelf)
GblRefCount	GblScanner_unref (GblScanner *pSelf)

Detailed Description

GblScanner, generic text tokenizer and lexer.

The main operation is to set the delimeter pattern (regex, powered by the TinyCRegex submodule), and then use GblScanner_peekToken() (and friends) to "view" the upcoming token without actually advancing the stream or do GblScanner_scanToken() (and friends) to take the next token and advance the buffer.

This basically lets GblScanner operate with a one-token lookahead at all time. If you need to inspect more tokens than just that without modifying the buffer state, that's when you would use GblScanner_pushCursor() to save your state, then peek/scan, and when you need to back up and restore an old state, use GblScanner_popCursor().

The main two operations are peek and scan, with their results being immediately stored in the public GblStringView variables GblScanner::next and GblScanner::token respectively. The variations of peek/scan are simply using GblStringView to handle type conversions for you, like GblScanner_scanUint32() for example, scans into GblScanner::token and then calls GblStringView_toUint32() on the token for you automatically.

Todo:

• continue testing type-specific peek/scan functions
• write own scanf() implementation that knows how many characters have been read
• store GblPattern now instead of GblStringRef
• peekPattern(), scanPattern()

Author

2023 Falco Girgis

Copyright

MIT License

Definition in file gimbal_scanner.h.

Macro Definition Documentation

GBL_SCANNER_TYPE

#define GBL_SCANNER_TYPE

Type UUID for GblScanner.

Definition at line 45 of file gimbal_scanner.h.

GBL_SCANNER

#define GBL_SCANNER

(

self

)

Function-style cast for GblInstance.

Definition at line 46 of file gimbal_scanner.h.

GBL_SCANNER_CLASS

#define GBL_SCANNER_CLASS

(

klass

)

Function-style cast for GblClass.

Definition at line 47 of file gimbal_scanner.h.

GBL_SCANNER_GET_CLASS

#define GBL_SCANNER_GET_CLASS

(

self

)

Get a GblScannerClass from GblInstance.

Definition at line 48 of file gimbal_scanner.h.

GBL_SCANNER_DELIMETERS_DEFAULT

#define GBL_SCANNER_DELIMETERS_DEFAULT

Default delimeters used with GblScanner for tokenizing the input stream.

Definition at line 52 of file gimbal_scanner.h.

Typedef Documentation

GBL_SCANNER_FLAGS

typedef GblEnum GBL_SCANNER_FLAGS

Definition at line 63 of file gimbal_scanner.h.

Enumeration Type Documentation

GBL_SCANNER_FLAGS

enum GBL_SCANNER_FLAGS

Extensible enum of flags used for GblScanner::status.

Enumerator
GBL_SCANNER_OK	No errors present.
GBL_SCANNER_EOF	End-of-file.
GBL_SCANNER_SCAN_ERROR	Failed to scan the previous token.
GBL_SCANNER_PEEK_ERROR	Failed to peek the previous token.
GBL_SCANNER_SKIP_ERROR	Failed previous skip call.
GBL_SCANNER_USER_STATE	Mask of non-error user flags.
GBL_SCANNER_USER_ERROR	Mask of error user flags.
GBL_SCANNER_STATE	Mask of all scanner state flags.
GBL_SCANNER_ERROR	Mask of all scanner error flags.

Definition at line 63 of file gimbal_scanner.h.

                                    {
    // Builtin Non-error state flags (bit 0)
    GBL_SCANNER_OK         = 0x00000000,    //!< No errors present
    GBL_SCANNER_EOF        = 0x00000001,    //!< End-of-file
    // Builtin Error flags (bits 29-31)
    GBL_SCANNER_SCAN_ERROR = 0x20000000,    //!< Failed to scan the previous token
    GBL_SCANNER_PEEK_ERROR = 0x40000000,    //!< Failed to peek the previous token
    GBL_SCANNER_SKIP_ERROR = 0x80000000,    //!< Failed previous skip call
    // User masks (bits 1-28)
    GBL_SCANNER_USER_STATE = 0x00fffff2,    //!< Mask of non-error user flags
    GBL_SCANNER_USER_ERROR = 0x1f000000,    //!< Mask of error user flags
    // Masks for combined state and error flags
    GBL_SCANNER_STATE      = 0x00ffffff,    //!< Mask of all scanner state flags
    GBL_SCANNER_ERROR      = 0xff000000,    //!< Mask of all scanner error flags
};

Function Documentation

GblScanner_create()

GblScanner * GblScanner_create	(	const char *	pStr,
		size_t	len
	)

Creates and returns a new GblScanner, setting its input to the given string with optional length.

GblScanner_ref()

GblScanner * GblScanner_ref

(

GblScanner *

pSelf

)

Acquires a reference to the given scanner, incrementing its reference count and returning its address.

GblScanner_unref()

GblRefCount GblScanner_unref

(

GblScanner *

pSelf

)

Releases a reference to the given scanner, returning the remaining count, and destroying the scanner when it hits 0.