A compilation unit scopes a group of related definitions. They are compiled and entered into the system in one atomic operation.

When a source file is being parsed, all definitions are part of a single compilation unit, unless the << parsing word is used to create nested compilation units.

Words defined in a compilation unit may not be called until the compilation unit is finished. The parser detects this case for parsing words and throws a staging-violation. Similarly, an attempt to use a macro from a word defined in the same compilation unit will throw a transform-expansion-error. Calling any other word from within its own compilation unit throws an undefined error.

This means that parsing words and macros generally cannot be used in the same source file as they are defined. There are two means of getting around this:
The simplest way is to split off the parsing words and macros into sub-vocabularies; perhaps suffixed by .syntax and .macros.
Alternatively, nested compilation units can be created using Parse time evaluation.

Parsing words which create new definitions at parse time will implicitly add them to the compilation unit of the current source file.

Code which creates new definitions at run time will need to explicitly create a compilation unit with a combinator. There is an additional combinator used by the parser to implement Parse time evaluation.
with-compilation-unit ( quot -- )

with-nested-compilation-unit ( quot -- )

Additional topics:
Compilation units internals