Class: CodeRay::TokenStream

Inherits:
Tokens show all
Defined in:
vendor/plugins/coderay-0.9.2/lib/coderay/tokens.rb

Overview

TokenStream

The TokenStream class is a fake Array without elements.

It redirects the method << to a block given at creation.

This allows scanners and Encoders to use streaming (no tokens are saved, the input is highlighted the same time it is scanned) with the same code.

See CodeRay.encode_stream and CodeRay.scan_stream

Constant Summary

Constants inherited from Tokens

ClassOfKind

Instance Attribute Summary

Instance Method Summary

Methods inherited from Tokens

#each, #each_text_token, #encode, #fix, #fix!, load, #method_missing, #optimize!, #split_into_lines, #split_into_lines!, #text, #to_s

Methods inherited from Array

#to_ber, #to_ber_appsequence, #to_ber_contextspecific, #to_ber_sequence, #to_ber_set, #to_csv

Methods included from Diffable

#diff, #patch, #replacenextlarger, #reverse_hash

Constructor Details

- (TokenStream) initialize(&block)

Creates a new TokenStream that calls block whenever its << method is called.

Example:

  require 'coderay'

  token_stream = CodeRay::TokenStream.new do |kind, text|
    puts 'kind: %s, text size: %d.' % [kind, text.size]
  end

  token_stream << [:regexp, '/\d+/']
  #-> kind: rexpexp, text size: 5.

Raises:

  • (ArgumentError)


311
312
313
314
315
# File 'vendor/plugins/coderay-0.9.2/lib/coderay/tokens.rb', line 311

def initialize &block
  raise ArgumentError, 'Block expected for streaming.' unless block
  @callback = block
  @size = 0
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method in the class CodeRay::Tokens

Instance Attribute Details

- (Object) size (readonly)

The Array is empty, but size counts the tokens given by <<.



295
296
297
# File 'vendor/plugins/coderay-0.9.2/lib/coderay/tokens.rb', line 295

def size
  @size
end

Instance Method Details

- (Object) <<(token)

Calls block with token and increments size.

Returns self.



320
321
322
323
324
# File 'vendor/plugins/coderay-0.9.2/lib/coderay/tokens.rb', line 320

def << token
  @callback.call(*token)
  @size += 1
  self
end

- (Object) dump

A TokenStream cannot be dumped. Use Tokens.

Raises:

  • (NotImplementedError)


333
334
335
# File 'vendor/plugins/coderay-0.9.2/lib/coderay/tokens.rb', line 333

def dump
  raise NotImplementedError, 'A TokenStream cannot be dumped.'
end

- (Object) optimize

A TokenStream cannot be optimized. Use Tokens.

Raises:

  • (NotImplementedError)


338
339
340
# File 'vendor/plugins/coderay-0.9.2/lib/coderay/tokens.rb', line 338

def optimize
  raise NotImplementedError, 'A TokenStream cannot be optimized.'
end

- (Boolean) stream?

Whether the object is a TokenStream.

Returns true.

Returns:

  • (Boolean)


290
291
292
# File 'vendor/plugins/coderay-0.9.2/lib/coderay/tokens.rb', line 290

def stream?
  true
end

- (Object) text_size

This method is not implemented due to speed reasons. Use Tokens.

Raises:

  • (NotImplementedError)


327
328
329
330
# File 'vendor/plugins/coderay-0.9.2/lib/coderay/tokens.rb', line 327

def text_size
  raise NotImplementedError,
    'This method is not implemented due to speed reasons.'
end