In file transfer, a “document definition” typically refers to a very specific, field-by-field description of a single document format (such as an ACH file) or single set of transaction data (such as EDI’s “997” Functional Acknowledgement).
Document definitions are used in transformation maps and can often be used outside of maps to validate the format of individual documents.
The best known example of a document definition language today is XML’s DTD (“Document Type Definition”).
Many transformation engines understand XML DTDs and some use standard transformation mechanisms like XSLT (“XML Transformations”). However most transformation engines depend on proprietary mapping formats (particularly for custom maps) that prevent much interoperability from one vendor to another.