Base class for tokens / ranges of tokens that can allow breaking within the tokens - for example, to avoid whitespace beyond the column limit, or to reflow text. More...

#include "/home/buildbot/as-worker-4/publish-doxygen-docs/llvm-project/clang/lib/Format/BreakableToken.h"

Inheritance diagram for clang::format::BreakableToken:

[legend]

Public Types
typedef std::pair< StringRef::size_type, unsigned >	Split
	Contains starting character index and length of split.

Public Member Functions
virtual	~BreakableToken ()

virtual unsigned	getLineCount () const =0
	Returns the number of lines in this token in the original code.

virtual unsigned	getRangeLength (unsigned LineIndex, unsigned Offset, StringRef::size_type Length, unsigned StartColumn) const =0
	Returns the number of columns required to format the text in the byte range [`Offset`, `Offset` `+` `Length`).

virtual unsigned	getRemainingLength (unsigned LineIndex, unsigned Offset, unsigned StartColumn) const
	Returns the number of columns required to format the text following the byte `Offset` in the line `LineIndex`, including potentially unbreakable sequences of tokens following after the end of the token.

virtual unsigned	getContentStartColumn (unsigned LineIndex, bool Break) const =0
	Returns the column at which content in line `LineIndex` starts, assuming no reflow.

virtual unsigned	getContentIndent (unsigned LineIndex) const
	Returns additional content indent required for the second line after the content at line `LineIndex` is broken.

virtual Split	getSplit (unsigned LineIndex, unsigned TailOffset, unsigned ColumnLimit, unsigned ContentStartColumn, const llvm::Regex &CommentPragmasRegex) const =0
	Returns a range (offset, length) at which to break the line at `LineIndex`, if previously broken at `TailOffset`.

virtual void	insertBreak (unsigned LineIndex, unsigned TailOffset, Split Split, unsigned ContentIndent, WhitespaceManager &Whitespaces) const =0
	Emits the previously retrieved `Split` via `Whitespaces`.

unsigned	getLengthAfterCompression (unsigned RemainingTokenColumns, Split Split) const
	Returns the number of columns needed to format `RemainingTokenColumns`, assuming that Split is within the range measured by `RemainingTokenColumns`, and that the whitespace in Split is reduced to a single space.

virtual void	compressWhitespace (unsigned LineIndex, unsigned TailOffset, Split Split, WhitespaceManager &Whitespaces) const =0
	Replaces the whitespace range described by `Split` with a single space.

virtual bool	supportsReflow () const
	Returns whether the token supports reflowing text.

virtual Split	getReflowSplit (unsigned LineIndex, const llvm::Regex &CommentPragmasRegex) const
	Returns a whitespace range (offset, length) of the content at `LineIndex` such that the content of that line is reflown to the end of the previous one.

virtual void	reflow (unsigned LineIndex, WhitespaceManager &Whitespaces) const
	Reflows the current line into the end of the previous one.

virtual bool	introducesBreakBeforeToken () const
	Returns whether there will be a line break at the start of the token.

virtual void	adaptStartOfLine (unsigned LineIndex, WhitespaceManager &Whitespaces) const
	Replaces the whitespace between `LineIndex-1` and `LineIndex`.

virtual Split	getSplitAfterLastLine (unsigned TailOffset) const
	Returns a whitespace range (offset, length) of the content at the last line that needs to be reformatted after the last line has been reformatted.

void	replaceWhitespaceAfterLastLine (unsigned TailOffset, Split SplitAfterLastLine, WhitespaceManager &Whitespaces) const
	Replaces the whitespace from `SplitAfterLastLine` on the last line after the last line has been formatted by performing a reformatting.

virtual void	updateNextToken (LineState &State) const
	Updates the next token of `State` to the next token after this one.

virtual void	updateAfterBroken (WhitespaceManager &Whitespaces) const
	Adds replacements that are needed when the token is broken.

Protected Member Functions
	BreakableToken (const FormatToken &Tok, bool InPPDirective, encoding::Encoding Encoding, const FormatStyle &Style)

Protected Attributes
const FormatToken &	Tok

const bool	InPPDirective

const encoding::Encoding	Encoding

const FormatStyle &	Style

Detailed Description

Base class for tokens / ranges of tokens that can allow breaking within the tokens - for example, to avoid whitespace beyond the column limit, or to reflow text.

Generally, a breakable token consists of logical lines, addressed by a line index. For example, in a sequence of line comments, each line comment is its own logical line; similarly, for a block comment, each line in the block comment is on its own logical line.

There are two methods to compute the layout of the token:

getRangeLength measures the number of columns needed for a range of text within a logical line, and
getContentStartColumn returns the start column at which we want the content of a logical line to start (potentially after introducing a line break).

The mechanism to adapt the layout of the breakable token is organised around the concept of a Split, which is a whitespace range that signifies a position of the content of a token where a reformatting might be done.

Operating with splits is divided into two operations:

getSplit, for finding a split starting at a position,
insertBreak, for executing the split using a whitespace manager.

There is a pair of operations that are used to compress a long whitespace range with a single space if that will bring the line length under the column limit:

getLineLengthAfterCompression, for calculating the size in columns of the line after a whitespace range has been compressed, and
compressWhitespace, for executing the whitespace compression using a whitespace manager; note that the compressed whitespace may be in the middle of the original line and of the reformatted line.

For tokens where the whitespace before each line needs to be also reformatted, for example for tokens supporting reflow, there are analogous operations that might be executed before the main line breaking occurs:

getReflowSplit, for finding a split such that the content preceding it needs to be specially reflown,
reflow, for executing the split using a whitespace manager,
introducesBreakBefore, for checking if reformatting the beginning of the content introduces a line break before it,
adaptStartOfLine, for executing the reflow using a whitespace manager.

For tokens that require the whitespace after the last line to be reformatted, for example in multiline jsdoc comments that require the trailing '*‍/' to be on a line of itself, there are analogous operations that might be executed after the last line has been reformatted:

getSplitAfterLastLine, for finding a split after the last line that needs to be reflown,
replaceWhitespaceAfterLastLine, for executing the reflow using a whitespace manager.

Definition at line 86 of file BreakableToken.h.

Member Typedef Documentation

◆ Split

typedef std::pair<StringRef::size_type, unsigned> clang::format::BreakableToken::Split

Contains starting character index and length of split.

Definition at line 89 of file BreakableToken.h.

Constructor & Destructor Documentation

◆ ~BreakableToken()

virtual clang::format::BreakableToken::~BreakableToken ( )

inlinevirtual

Definition at line 91 of file BreakableToken.h.

◆ BreakableToken()

clang::format::BreakableToken::BreakableToken	(	const FormatToken &	Tok,
		bool	InPPDirective,
		encoding::Encoding	Encoding,
		const FormatStyle &	Style
	)

inlineprotected

Definition at line 236 of file BreakableToken.h.

Member Function Documentation

◆ adaptStartOfLine()

virtual void clang::format::BreakableToken::adaptStartOfLine	(	unsigned	LineIndex,
		WhitespaceManager &	Whitespaces
	)		const

inlinevirtual

Replaces the whitespace between LineIndex-1 and LineIndex.

Reimplemented in clang::format::BreakableBlockComment, and clang::format::BreakableLineCommentSection.

Definition at line 203 of file BreakableToken.h.

◆ compressWhitespace()

virtual void clang::format::BreakableToken::compressWhitespace	(	unsigned	LineIndex,
		unsigned	TailOffset,
		Split	Split,
		WhitespaceManager &	Whitespaces
	)		const

pure virtual

Replaces the whitespace range described by Split with a single space.

Implemented in clang::format::BreakableStringLiteral, and clang::format::BreakableComment.

◆ getContentIndent()

virtual unsigned clang::format::BreakableToken::getContentIndent ( unsigned LineIndex ) const

inlinevirtual

Returns additional content indent required for the second line after the content at line LineIndex is broken.

Reimplemented in clang::format::BreakableBlockComment.

Definition at line 146 of file BreakableToken.h.

◆ getContentStartColumn()

virtual unsigned clang::format::BreakableToken::getContentStartColumn	(	unsigned	LineIndex,
		bool	Break
	)		const

pure virtual

Returns the column at which content in line LineIndex starts, assuming no reflow.

If Break is true, returns the column at which the line should start after the line break. If Break is false, returns the column at which the line itself will start.

Implemented in clang::format::BreakableStringLiteral, clang::format::BreakableStringLiteralUsingOperators, clang::format::BreakableBlockComment, and clang::format::BreakableLineCommentSection.

◆ getLengthAfterCompression()

unsigned clang::format::BreakableToken::getLengthAfterCompression	(	unsigned	RemainingTokenColumns,
		Split	Split
	)		const

Returns the number of columns needed to format RemainingTokenColumns, assuming that Split is within the range measured by RemainingTokenColumns, and that the whitespace in Split is reduced to a single space.

Definition at line 228 of file BreakableToken.cpp.

◆ getLineCount()

virtual unsigned clang::format::BreakableToken::getLineCount ( ) const

pure virtual

Returns the number of lines in this token in the original code.

Implemented in clang::format::BreakableStringLiteral, and clang::format::BreakableComment.

Referenced by replaceWhitespaceAfterLastLine().

◆ getRangeLength()

virtual unsigned clang::format::BreakableToken::getRangeLength	(	unsigned	LineIndex,
		unsigned	Offset,
		StringRef::size_type	Length,
		unsigned	StartColumn
	)		const

pure virtual

Returns the number of columns required to format the text in the byte range [Offset, Offset + Length).

Offset is the byte offset from the start of the content of the line at LineIndex.

StartColumn is the column at which the text starts in the formatted file, needed to compute tab stops correctly.

Implemented in clang::format::BreakableStringLiteral, clang::format::BreakableBlockComment, and clang::format::BreakableLineCommentSection.

Referenced by getRemainingLength().

◆ getReflowSplit()

virtual Split clang::format::BreakableToken::getReflowSplit	(	unsigned	LineIndex,
		const llvm::Regex &	CommentPragmasRegex
	)		const

inlinevirtual

Returns a whitespace range (offset, length) of the content at LineIndex such that the content of that line is reflown to the end of the previous one.

Returning (StringRef::npos, 0) indicates reflowing is not possible.

The range will include any whitespace preceding the specified line's content.

If the split is not contained within one token, for example when reflowing line comments, returns (0, <length>).

Reimplemented in clang::format::BreakableBlockComment, and clang::format::BreakableLineCommentSection.

Definition at line 189 of file BreakableToken.h.

◆ getRemainingLength()

virtual unsigned clang::format::BreakableToken::getRemainingLength	(	unsigned	LineIndex,
		unsigned	Offset,
		unsigned	StartColumn
	)		const

inlinevirtual

Returns the number of columns required to format the text following the byte Offset in the line LineIndex, including potentially unbreakable sequences of tokens following after the end of the token.

Offset is the byte offset from the start of the content of the line at LineIndex.

StartColumn is the column at which the text starts in the formatted file, needed to compute tab stops correctly.

For breakable tokens that never use extra space at the end of a line, this is equivalent to getRangeLength with a Length of StringRef::npos.

Reimplemented in clang::format::BreakableStringLiteral, clang::format::BreakableStringLiteralUsingOperators, and clang::format::BreakableBlockComment.

Definition at line 120 of file BreakableToken.h.

References getRangeLength().

◆ getSplit()

virtual Split clang::format::BreakableToken::getSplit	(	unsigned	LineIndex,
		unsigned	TailOffset,
		unsigned	ColumnLimit,
		unsigned	ContentStartColumn,
		const llvm::Regex &	CommentPragmasRegex
	)		const

pure virtual

Returns a range (offset, length) at which to break the line at LineIndex, if previously broken at TailOffset.

If possible, do not violate ColumnLimit, assuming the text starting at TailOffset in the token is formatted starting at ContentStartColumn in the reformatted file.

Implemented in clang::format::BreakableStringLiteral, clang::format::BreakableComment, and clang::format::BreakableBlockComment.

◆ getSplitAfterLastLine()

virtual Split clang::format::BreakableToken::getSplitAfterLastLine ( unsigned TailOffset ) const

inlinevirtual

Returns a whitespace range (offset, length) of the content at the last line that needs to be reformatted after the last line has been reformatted.

A result having offset == StringRef::npos means that no reformat is necessary.

Reimplemented in clang::format::BreakableBlockComment.

Definition at line 212 of file BreakableToken.h.

◆ insertBreak()

virtual void clang::format::BreakableToken::insertBreak	(	unsigned	LineIndex,
		unsigned	TailOffset,
		Split	Split,
		unsigned	ContentIndent,
		WhitespaceManager &	Whitespaces
	)		const

pure virtual

Emits the previously retrieved Split via Whitespaces.

Implemented in clang::format::BreakableStringLiteral, clang::format::BreakableStringLiteralUsingOperators, clang::format::BreakableBlockComment, and clang::format::BreakableLineCommentSection.

Referenced by replaceWhitespaceAfterLastLine().

◆ introducesBreakBeforeToken()

virtual bool clang::format::BreakableToken::introducesBreakBeforeToken ( ) const

inlinevirtual

Returns whether there will be a line break at the start of the token.

Reimplemented in clang::format::BreakableBlockComment.

Definition at line 200 of file BreakableToken.h.

◆ reflow()

virtual void clang::format::BreakableToken::reflow	(	unsigned	LineIndex,
		WhitespaceManager &	Whitespaces
	)		const

inlinevirtual

Reflows the current line into the end of the previous one.

Reimplemented in clang::format::BreakableBlockComment, and clang::format::BreakableLineCommentSection.

Definition at line 195 of file BreakableToken.h.

◆ replaceWhitespaceAfterLastLine()

void clang::format::BreakableToken::replaceWhitespaceAfterLastLine	(	unsigned	TailOffset,
		Split	SplitAfterLastLine,
		WhitespaceManager &	Whitespaces
	)		const

inline

Replaces the whitespace from SplitAfterLastLine on the last line after the last line has been formatted by performing a reformatting.

Definition at line 218 of file BreakableToken.h.

References getLineCount(), and insertBreak().

◆ supportsReflow()

virtual bool clang::format::BreakableToken::supportsReflow ( ) const

inlinevirtual

Returns whether the token supports reflowing text.

Reimplemented in clang::format::BreakableComment.

Definition at line 176 of file BreakableToken.h.

◆ updateAfterBroken()

virtual void clang::format::BreakableToken::updateAfterBroken ( WhitespaceManager & Whitespaces ) const

inlinevirtual

Adds replacements that are needed when the token is broken.

Such as wrapping a JavaScript string in parentheses after it gets broken with plus signs.

Reimplemented in clang::format::BreakableStringLiteralUsingOperators.

Definition at line 233 of file BreakableToken.h.

◆ updateNextToken()

virtual void clang::format::BreakableToken::updateNextToken ( LineState & State ) const

inlinevirtual

Updates the next token of State to the next token after this one.

This can be used when this token manages a set of underlying tokens as a unit and is responsible for the formatting of the them.

Reimplemented in clang::format::BreakableLineCommentSection.

Definition at line 228 of file BreakableToken.h.

Member Data Documentation

◆ Encoding

const encoding::Encoding clang::format::BreakableToken::Encoding

protected

Definition at line 243 of file BreakableToken.h.

◆ InPPDirective

const bool clang::format::BreakableToken::InPPDirective

protected

Definition at line 242 of file BreakableToken.h.

Referenced by clang::format::BreakableBlockComment::adaptStartOfLine(), clang::format::BreakableStringLiteral::insertBreak(), clang::format::BreakableStringLiteralUsingOperators::insertBreak(), clang::format::BreakableBlockComment::insertBreak(), clang::format::BreakableLineCommentSection::insertBreak(), clang::format::BreakableBlockComment::reflow(), and clang::format::BreakableStringLiteralUsingOperators::updateAfterBroken().

◆ Style

const FormatStyle& clang::format::BreakableToken::Style

protected

Definition at line 244 of file BreakableToken.h.

◆ Tok

const FormatToken& clang::format::BreakableToken::Tok

protected

Definition at line 241 of file BreakableToken.h.

Referenced by clang::format::BreakableBlockComment::BreakableBlockComment(), clang::format::BreakableLineCommentSection::BreakableLineCommentSection(), clang::format::BreakableStringLiteral::BreakableStringLiteral(), clang::format::BreakableStringLiteral::insertBreak(), clang::format::BreakableStringLiteralUsingOperators::insertBreak(), clang::format::BreakableBlockComment::mayReflow(), clang::format::BreakableLineCommentSection::mayReflow(), clang::format::BreakableComment::tokenAt(), and clang::format::BreakableStringLiteralUsingOperators::updateAfterBroken().

The documentation for this class was generated from the following files:

lib/Format/BreakableToken.h
lib/Format/BreakableToken.cpp

Public Types

Public Member Functions

Protected Member Functions

Protected Attributes

Detailed Description

Member Typedef Documentation

◆ Split

Constructor & Destructor Documentation

◆ ~BreakableToken()

◆ BreakableToken()

Member Function Documentation

◆ adaptStartOfLine()

◆ compressWhitespace()

◆ getContentIndent()

◆ getContentStartColumn()

◆ getLengthAfterCompression()

◆ getLineCount()

◆ getRangeLength()

◆ getReflowSplit()

◆ getRemainingLength()

◆ getSplit()

◆ getSplitAfterLastLine()

◆ insertBreak()

◆ introducesBreakBeforeToken()

◆ reflow()

◆ replaceWhitespaceAfterLastLine()

◆ supportsReflow()

◆ updateAfterBroken()

◆ updateNextToken()

Member Data Documentation

◆ Encoding

◆ InPPDirective

◆ Style

◆ Tok