Class: CodeRay::TokenStream
Overview
TokenStream
The TokenStream class is a fake Array without elements.
It redirects the method << to a block given at creation.
This allows scanners and Encoders to use streaming (no tokens are saved, the input is highlighted the same time it is scanned) with the same code.
See CodeRay.encode_stream and CodeRay.scan_stream
Constant Summary
Constants inherited from Tokens
Instance Attribute Summary
-
- (Object) size
readonly
The Array is empty, but size counts the tokens given by <<.
Instance Method Summary
-
- (Object) <<(token)
Calls block with token and increments size.
-
- (Object) dump
A TokenStream cannot be dumped.
-
- (TokenStream) initialize(&block)
constructor
Creates a new TokenStream that calls block whenever its << method is called.
-
- (Object) optimize
A TokenStream cannot be optimized.
-
- (Boolean) stream?
Whether the object is a TokenStream.
-
- (Object) text_size
This method is not implemented due to speed reasons.
Methods inherited from Tokens
#each, #each_text_token, #encode, #fix, #fix!, load, #method_missing, #optimize!, #split_into_lines, #split_into_lines!, #text, #to_s
Methods inherited from Array
#to_ber, #to_ber_appsequence, #to_ber_contextspecific, #to_ber_sequence, #to_ber_set, #to_csv
Methods included from Diffable
#diff, #patch, #replacenextlarger, #reverse_hash
Constructor Details
- (TokenStream) initialize(&block)
Creates a new TokenStream that calls block whenever its << method is called.
Example:
require 'coderay' token_stream = CodeRay::TokenStream.new do |kind, text| puts 'kind: %s, text size: %d.' % [kind, text.size] end token_stream << [:regexp, '/\d+/'] #-> kind: rexpexp, text size: 5.
311 312 313 314 315 |
# File 'vendor/plugins/coderay-0.9.2/lib/coderay/tokens.rb', line 311 def initialize &block raise ArgumentError, 'Block expected for streaming.' unless block @callback = block @size = 0 end |
Dynamic Method Handling
This class handles dynamic methods through the method_missing method in the class CodeRay::Tokens
Instance Attribute Details
- (Object) size (readonly)
The Array is empty, but size counts the tokens given by <<.
295 296 297 |
# File 'vendor/plugins/coderay-0.9.2/lib/coderay/tokens.rb', line 295 def size @size end |
Instance Method Details
- (Object) <<(token)
Calls block with token and increments size.
Returns self.
320 321 322 323 324 |
# File 'vendor/plugins/coderay-0.9.2/lib/coderay/tokens.rb', line 320 def << token @callback.call(*token) @size += 1 self end |
- (Object) dump
A TokenStream cannot be dumped. Use Tokens.
333 334 335 |
# File 'vendor/plugins/coderay-0.9.2/lib/coderay/tokens.rb', line 333 def dump raise NotImplementedError, 'A TokenStream cannot be dumped.' end |
- (Object) optimize
A TokenStream cannot be optimized. Use Tokens.
338 339 340 |
# File 'vendor/plugins/coderay-0.9.2/lib/coderay/tokens.rb', line 338 def optimize raise NotImplementedError, 'A TokenStream cannot be optimized.' end |
- (Boolean) stream?
Whether the object is a TokenStream.
Returns true.
290 291 292 |
# File 'vendor/plugins/coderay-0.9.2/lib/coderay/tokens.rb', line 290 def stream? true end |
- (Object) text_size
This method is not implemented due to speed reasons. Use Tokens.
327 328 329 330 |
# File 'vendor/plugins/coderay-0.9.2/lib/coderay/tokens.rb', line 327 def text_size raise NotImplementedError, 'This method is not implemented due to speed reasons.' end |