Support Bash

[ghost]

This has pretty much already been done for you, just need to pull in the files. I have been working with Petr Kovář to fix a couple of bugs.

github.com/pejuko/coderay_bash

[korny]

Thank you for working on this!

I collected a few historical resources for reference, since there have been efforts to create a Bash scanner in the past:

• A very old open ticket on the CodeRay Redmine project for a Bash scanner
• Vincent Landgraf wrote a Bash scanner
• Anh Ky Huynh also wrote one
• And now, finally, Petr Kovář's and your Bash scanner, the only one compatible with CodeRay 1.0

I want to include it in the next minor release of CodeRay. To do that, we need example code - lots of it, since Bash is not a simple language. So, in addition to the test files included in the coderay_bash repository, I added all Bash scripts in rvm.

The bash-scanner branch includes your scanner, so I can test it. Unfortunately, the large rvm file breaks it:

             example  456.5 KiB incremental, shuffled, complete...scanning.../Users/murphy/ruby/coderay/lib/coderay/scanners/bash.rb:246: warning: character class has duplicated range: /[^\)\$RubyWrapper\\]+/
/Users/murphy/ruby/coderay/lib/coderay/scanners/bash.rb:246: warning: character class has duplicated range: /[^\)\$RubyWrapper\\]+/
/Users/murphy/ruby/coderay/lib/coderay/scanners/bash.rb:246: warning: character class has duplicated range: /[^\)\$RubyWrapper\\]+/
/Users/murphy/ruby/coderay/lib/coderay/scanners/bash.rb:246: warning: character class has duplicated range: /[^\)\$RubyWrapper\\]+/
/Users/murphy/ruby/coderay/lib/coderay/scanners/bash.rb:246: warning: character class has duplicated range: /[^\)\$RubyWrapper\\]+/
/Users/murphy/ruby/coderay/lib/coderay/scanners/bash.rb:246: warning: character class has duplicated range: /[^\)\$RubyWrapper\\]+/
/Users/murphy/ruby/coderay/lib/coderay/scanners/bash.rb:246: warning: character class has duplicated range: /[^\)\$RubyWrapper\\]+/
/Users/murphy/ruby/coderay/lib/coderay/scanners/bash.rb:246: warning: character class has duplicated range: /[^\)\$RubyWrapper\\]+/
/Users/murphy/ruby/coderay/lib/coderay/scanners/bash.rb:246: warning: character class has duplicated range: /[^\)\$RubyWrapper\\]+/
/Users/murphy/ruby/coderay/lib/coderay/scanners/bash.rb:246: warning: character class has duplicated range: /[^\)\$RubyWrapper\\]+/
/Users/murphy/ruby/coderay/lib/coderay/scanners/bash.rb:246: warning: character class has duplicated range: /[^\)\$RubyWrapper\\]+/
/Users/murphy/ruby/coderay/lib/coderay/scanners/bash.rb:246: warning: character class has duplicated range: /[^\)\$RubyWrapper\\]+/
/Users/murphy/ruby/coderay/lib/coderay/scanners/bash.rb:246: warning: character class has duplicated range: /[^\)\$RubyWrapper\\]+/
/Users/murphy/ruby/coderay/lib/coderay/scanners/bash.rb:246: warning: character class has duplicated range: /[^\)\$RubyWrapper\\]+/
encoding...directive(#!/usr/bin/env bash)plain(
)ident(RVM) ident(is) ident(the) ident(Ruby) ident(enVironment) ident(Manager) operator(()ident(rvm)(\))plain(.)plain(
)plain(
)reserved(function) ident(bundle_not_found)operator(()(\))plain(
)reserved({)plain(
)  method(printf) string<delimiter(")content(%b)delimiter(")> string<delimiter(")shell<delimiter($()content(tput setaf 1)delimiter(\))>content(ERROR: Gem bundler is not installed, run )content(\\`)content(gem install bundler)content(\\`)content( first.)shell<delimiter($()content(tput sgr0)delimiter(\))>content(\\n)delimiter(")>plain(
)  method(exit) integer(127)plain(
)reserved(})plain(
)plain(
)comment(# in rvm warn about missing gem)plain(
)reserved(if) reserved([)reserved([) operator(-)ident(n) string<delimiter(")instance_variable(${GEM_HOME:-})delimiter(")> reserved(])reserved(])plain(
)reserved(then)plain(
)  ident(bundle_not_found)plain(
)reserved(else)plain(
)  instance_variable(current_bundle)operator(=)string<delimiter(")shell<delimiter($()content(dirname )shell<delimiter($()content(which )predefined_constant($0)delimiter(\))>delimiter(\))>delimiter(")>plain(
)  method(export) predefined(PATH)plain(
)  predefined(PATH)operator(=)string<delimiter(")content(:)instance_variable(${PATH})content(:)delimiter(")>plain(
)  predefined(PATH)operator(=)string<delimiter(")instance_variable(${PATH//:${current_bundle})content(:/:})delimiter(")>plain(
)  predefined(PATH)operator(=)string<delimiter(")instance_variable(${PATH#:})delimiter(")>plain(
)  predefined(PATH)operator(=)string<delimiter(")instance_variable(${PATH%:})delimiter(")>plain(
)  reserved(if) reserved([)reserved([) operator(-)ident(n) string<delimiter(")instance_variable(${current_bundle})delimiter(")> reserved(])reserved(]) bin(&)bin(&) method(builtin) method(command) operator(-)ident(v) ident(bundle) bin(>)plain(/)ident(dev)plain(/)ident(null) integer(2)bin(>)bin(&)integer(1)plain(
)  reserved(then)plain(
)    method(builtin) method(command) ident(bundle) string<delimiter(")predefined_constant($@)delimiter(")>plain(
)  reserved(else)plain(
)    ident(bundle_not_found)plain(
)  reserved(fi)plain(
)reserved(fi)plain(
)directive(#!/usr/bin/env bash)plain(
)plain(
)reserved(function) ident(rake_not_found)operator(()(\))plain(
)reserved({)plain(
)  method(printf) string<delimiter(")content(%b)delimiter(")> string<delimiter(")shell<delimiter($()content(tput setaf 1)delimiter(\))>content(ERROR: Gem rake is not installed, run )content(\\`)content(gem install rake)content(\\`)content( first.)shell<delimiter($()content(tput sgr0)delimiter(\))>content(\\n)delimiter(")>plain(
)  method(exit) integer(127)plain(
)reserved(})plain(
)plain(
)comment(# in rvm warn about missing gem)plain(
)reserved(if) reserved([)reserved([) operator(-)ident(n) string<delimiter(")instance_variable(${GEM_HOME:-})delimiter(")> reserved(])reserved(])plain(
)reserved(then)plain(
)  ident(rake_not_found)plain(
)reserved(else)plain(
)  instance_variable(current_rake)operator(=)string<delimiter(")shell<delimiter($()content(dirname )shell<delimiter($()content(which )predefined_constant($0)delimiter(\))>delimiter(\))>delimiter(")>plain(
)  method(export) predefined(PATH)plain(
)  predefined(PATH)operator(=)string<delimiter(")content(:)instance_variable(${PATH})content(:)delimiter(")>plain(
)  predefined(PATH)operator(=)string<delimiter(")instance_variable(${PATH//:${current_rake})content(:/:})delimiter(")>plain(
)  predefined(PATH)operator(=)string<delimiter(")instance_variable(${PATH#:})delimiter(")>plain(
)  predefined(PATH)operator(=)string<delimiter(")instance_variable(${PATH%:})delimiter(")>plain(
)  reserved(if) reserved([)reserved([) operator(-)ident(n) string<delimiter(")instance_variable(${current_rake})delimiter(")> reserved(])reserved(]) bin(&)bin(&) method(builtin) method(command) operator(-)ident(v) ident(rake) bin(>)plain(/)ident(dev)plain(/)ident(null) integer(2)bin(>)bin(&)integer(1)plain(
)  reserved(then)plain(
)    method(builtin) method(command) ident(rake) string<delimiter(")predefined_constant($@)delimiter(")>plain(
)  reserved(else)plain(
)    ident(rake_not_found)plain(
)  reserved(fi)plain(
)reserved(fi)plain(
)directive(#!/usr/bin/env bash)plain(
)plain(
)reserved(if) operator(()operator(() instance_variable(${rvm_ignore_rvmrc:=0}) operator(=)operator(=) integer(0) global_variable(\)\))plain(
)reserved(then)plain(
)  method(declare) ident(rvmrc)plain(
)plain(
)  instance_variable(rvm_rvmrc_files)operator(=)operator(()string<delimiter(")content(/etc/rvmrc)delimiter(")> string<delimiter(")instance_variable($HOME)content(/.rvmrc)delimiter(")>(\))plain(
)  reserved(if) reserved([)reserved([) operator(-)ident(n) string<delimiter(")instance_variable(${rvm_prefix:-})delimiter(")> reserved(])reserved(]) bin(&)bin(&) reserved(!) reserved([)reserved([) string<delimiter(")instance_variable($HOME)content(/.rvmrc)delimiter(")> operator(-)ident(ef) string<delimiter(")instance_variable(${rvm_prefix})content(/.rvmrc)delimiter(")> reserved(])reserved(])plain(
)     reserved(then) instance_variable(rvm_rvmrc_files)operator(+=)operator(() string<delimiter(")instance_variable(${rvm_prefix})content(/.rvmrc)delimiter(")> (\))plain(
)  reserved(fi)plain(
)  
  reserved(for) ident(rvmrc) reserved(in) string<delimiter(")instance_variable(${)instance_variable(rvm_rvmrc_files)operator([)key(@)operator(])instance_variable(})delimiter(")>plain(
)  reserved(do)plain(
)    reserved(if) reserved([)reserved([) operator(-)ident(f) string<delimiter(")instance_variable($rvmrc)delimiter(")> reserved(])reserved(])plain(
)    reserved(then)plain(
)      reserved(if) instance_variable(GREP_OPTIONS)operator(=)string<delimiter(")delimiter(")> plain(\\g)ident(rep) string('^\\s*rvm .*$') string<delimiter(")instance_variable($rvmrc)delimiter(")> bin(>)plain(/)ident(dev)plain(/)ident(null) integer(2)bin(>)bin(&)integer(1)plain(
)      reserved(then)plain(
)        method(printf) string<delimiter(")content(%b)delimiter(")> string<delimiter(")plain(
)content(  Error:
    )instance_variable($rvmrc)content( is for rvm settings only.
    rvm CLI may NOT be called from within )instance_variable($rvmrc)content(.
    Skipping the loading of )instance_variable($rvmrc)plain(
)delimiter(")>plain(
)        method(exit) integer(1)plain(
)      reserved(else)plain(
)        reserved(source "$rvmrc")plain(
)      reserved(fi)plain(
)    reserved(fi)plain(
)  reserved(done)plain(
)  method(unset) ident(rvm_rvmrc_files)plain(
)  method(unset) ident(rvmrc)plain(
)reserved(fi)plain(
)plain(
)method(export) ident(rvm_path)plain(
)reserved(if) reserved([)reserved([) operator(-)ident(z) string<delimiter(")instance_variable(${rvm_path:-})delimiter(")> reserved(])reserved(])plain(
)reserved(then)plain(
)  reserved(if) operator(()operator(() predefined(UID) operator(=)operator(=) integer(0) global_variable(\)\)) bin(&)bin(&) reserved([)reserved([) operator(-)ident(d) string<delimiter(")content(/usr/local/rvm)delimiter(")> reserved(])reserved(])plain(
)  reserved(then) instance_variable(rvm_path)operator(=)string<delimiter(")content(/usr/local/rvm)delimiter(")>plain(
)  reserved(elif) reserved([)reserved([) operator(-)ident(d) string<delimiter(")instance_variable(${HOME})content(/.rvm)delimiter(")> reserved(])reserved(])plain(
)  reserved(then) instance_variable(rvm_path)operator(=)string<delimiter(")instance_variable(${HOME})content(/.rvm)delimiter(")>plain(
)  reserved(elif) reserved([)reserved([) operator(-)ident(d) string<delimiter(")content(/usr/local/rvm)delimiter(")> reserved(])reserved(])plain(
)  reserved(then) instance_variable(rvm_path)operator(=)string<delimiter(")content(/usr/local/rvm)delimiter(")>plain(
)  reserved(else) method(echo) string<delimiter(")content(Can't find rvm install!)delimiter(")> integer(1)bin(>)bin(&)integer(2) delimiter(;) method(exit) integer(1)plain(
)  reserved(fi)plain(
)reserved(fi)plain(
)plain(
)comment(# allow disabling check temporary)plain(
)reserved(:) ident(rvm_is_not_a_shell_function)reserved(:)instance_variable(${rvm_is_not_a_shell_function:=1})plain(
)plain(
)comment(# if to prevent fork-bomb)plain(
)reserved(if) reserved(source "${rvm_scripts_path:="$rvm_path/scripts"}/rvm")plain(
)reserved(then)plain(
)  ident(rvm) string<delimiter(")predefined_constant($@)delimiter(")>plain(
)reserved(else)plain(
)  method(echo) string<delimiter(")content(Error sourcing RVM!)delimiter(")>  integer(1)bin(>)bin(&)integer(2)plain(
)  method(exit) integer(1)plain(
)reserved(fi)plain(
)directive(#!/usr/bin/env bash)plain(
)plain(
)method(export) predefined(HOME)operator(=)string<delimiter(")instance_variable(${HOME%%+(\\/\)})delimiter(")>comment( # Remove trailing slashes if they exist on HOME)plain(
)plain(
)reserved(if) operator(()operator(() instance_variable(${rvm_ignore_rvmrc:=0}) operator(=)operator(=) integer(0) global_variable(\)\))plain(
)reserved(then)plain(
)  reserved(for) ident(rvmrc) reserved(in) plain(/)ident(etc)plain(/)ident(rvmrc) string<delimiter(")instance_variable($HOME)content(/.rvmrc)delimiter(")>plain(
)  reserved(do)plain(
)    reserved(if) reserved([)reserved([) operator(-)ident(f) string<delimiter(")instance_variable($rvmrc)delimiter(")> reserved(])reserved(])plain(
)    reserved(then)plain(
)      reserved(if) instance_variable(GREP_OPTIONS)operator(=)string<delimiter(")delimiter(")> plain(\\g)ident(rep) string('^\\s*rvm .*$') string<delimiter(")instance_variable($rvmrc)delimiter(")> bin(>)plain(/)ident(dev)plain(/)ident(null) integer(2)bin(>)bin(&)integer(1)plain(
)      reserved(then)plain(
)        method(printf) string<delimiter(")content(%b)delimiter(")> string<delimiter(")content(\\n)content(Error: )instance_variable($rvmrc)content( is for rvm settings only.)content(\\n)content(rvm CLI may NOT be called from within )instance_variable($rvmrc)content(. )content(\\n)content(Skipping the loading of )instance_variable($rvmrc)delimiter(")>plain(
)        method(return) integer(1)plain(
)      reserved(else)plain(
)        reserved(source "$rvmrc")plain(
)      reserved(fi)plain(
)    reserved(fi)plain(
)  reserved(done)plain(
)reserved(fi)plain(
)plain(
)method(export) ident(rvm_path)plain(
)reserved(if) reserved([)reserved([) operator(-)ident(z) string<delimiter(")instance_variable(${rvm_path:-})delimiter(")> reserved(])reserved(])plain(
)reserved(then)plain(
)  reserved(if) operator(()operator(() predefined(UID) operator(=)operator(=) integer(0) global_variable(\)\)) bin(&)bin(&) reserved([)reserved([) operator(-)ident(d) string<delimiter(")content(/usr/local/rvm)delimiter(")> reserved(])reserved(])plain(
)  reserved(then) instance_variable(rvm_path)operator(=)string<delimiter(")content(/usr/local/rvm)delimiter(")>plain(
)  reserved(elif) reserved([)reserved([) operator(-)ident(d) string<delimiter(")instance_variable(${HOME})content(/.rvm)delimiter(")> reserved(])reserved(])plain(
)  reserved(then) instance_variable(rvm_path)operator(=)string<delimiter(")instance_variable(${HOME})content(/.rvm)delimiter(")>plain(
)  reserved(elif) reserved([)reserved([) operator(-)ident(d) string<delimiter(")content(/usr/local/rvm)delimiter(")> reserved(])reserved(])plain(
)  reserved(then) instance_variable(rvm_path)operator(=)string<delimiter(")content(/usr/local/rvm)delimiter(")>plain(
)  reserved(else) method(echo) string<delimiter(")content(Can't find rvm install!)delimiter(")> integer(1)bin(>)bin(&)integer(2) delimiter(;) method(exit) integer(1)plain(
)  reserved(fi)plain(
)reserved(fi)plain(
)plain(
)ident(true) instance_variable(${rvm_scripts_path:="$rvm_path/scripts"})plain(
)ident(true) instance_variable(${rvm_environments_path:="$rvm_path/environments"})plain(
)plain(
)reserved(if) reserved([)reserved([) operator(-)ident(n) string<delimiter(")instance_variable($rvm_path)delimiter(")> bin(&)bin(&) operator(-)ident(s) string<delimiter(")instance_variable($rvm_scripts_path)content(/rvm)delimiter(")> reserved(])reserved(])delimiter(;) reserved(then)plain(
)  reserved(source "$rvm_scripts_path/rvm" > /dev/null 2>&1)plain(
)plain(
)reserved(elif) reserved([)reserved([) operator(-)ident(s) string<delimiter(")instance_variable($HOME)content(/.rvm/scripts/rvm)delimiter(")> reserved(])reserved(])delimiter(;) reserved(then)plain(
)  reserved(source "$HOME/.rvm/scripts/rvm" > /dev/null 2>&1)plain(
)plain(
)reserved(elif) reserved([)reserved([) operator(-)ident(s) string<delimiter(")content(/usr/local/rvm/scripts/rvm)delimiter(")> reserved(])reserved(])delimiter(;) reserved(then)plain(
)  reserved(source "/usr/local/rvm/scripts/rvm" > /dev/null 2>&1)plain(
)plain(
)reserved(else)plain(
)  method(echo) string<delimiter(")content(Unable to detect rvm, please manually set the rvm_path env variable.)delimiter(")> bin(>)bin(&)integer(2)plain(
)  method(exit) integer(1)plain(
)reserved(fi)plain(
)plain(
)reserved([)reserved([) operator(-)ident(s) string<delimiter(")instance_variable($rvm_environments_path)content(/default)delimiter(")> reserved(])reserved(]) bin(&)bin(&) reserved(source "$rvm_environments_path/default")plain(
)plain(
)instance_variable(rvm_promptless)operator(=)integer(1) ident(rvm) ident(rvmrc) ident(load) bin(>) plain(/)ident(dev)plain(/)ident(null) integer(2)bin(>)bin(&)integer(1)plain(
)plain(
)method(exec) ident(ruby) string<delimiter(")predefined_constant($@)delimiter(")>plain(
)directive(#!/usr/bin/env bash)plain(
)plain(
)method(export) predefined(HOME)operator(=)string<delimiter(")instance_variable(${HOME%%+(\\/\)})delimiter(")>comment( # Remove trailing slashes if they exist on HOME)plain(
)plain(
)reserved(if) operator(()operator(() instance_variable(${rvm_ignore_rvmrc:=0}) operator(=)operator(=) integer(0) global_variable(\)\))plain(
)reserved(then)plain(
)  reserved(for) ident(rvmrc) reserved(in) plain(/)ident(etc)plain(/)ident(rvmrc) string<delimiter(")instance_variable($HOME)content(/.rvmrc)delimiter(")>plain(
)  reserved(do)plain(
)    reserved(if) reserved([)reserved([) operator(-)ident(f) string<delimiter(")instance_variable($rvmrc)delimiter(")> reserved(])reserved(])plain(
)    reserved(then)plain(
)      reserved(if) instance_variable(GREP_OPTIONS)operator(=)string<delimiter(")delimiter(")> plain(\\g)ident(rep) string('^\\s*rvm .*$') string<delimiter(")instance_variable($rvmrc)delimiter(")> bin(>)plain(/)ident(dev)plain(/)ident(null) integer(2)bin(>)bin(&)integer(1)plain(
)      reserved(then)plain(
)        method(printf) string<delimiter(")content(%b)delimiter(")> string<delimiter(")content(\\n)content(  Error:
    )instance_variable($rvmrc)content( is for rvm settings only.
    rvm CLI may NOT be called from within )instance_variable($rvmrc)content(.
    Skipping the loading of )instance_variable($rvmrc)plain(
)delimiter(")>plain(
)        method(exit) integer(1)plain(
)      reserved(else)plain(
)        reserved(source "$rvmrc")plain(
)      reserved(fi)plain(
)    reserved(fi)plain(
)  reserved(done)plain(
)reserved(fi)plain(
)plain(
)method(export) ident(rvm_path)plain(
)reserved(if) reserved([)reserved([) operator(-)ident(z) string<delimiter(")instance_variable(${rvm_path:-})delimiter(")> reserved(])reserved(])plain(
)reserved(then)plain(
)  reserved(if) operator(()operator(() predefined(UID) operator(=)operator(=) integer(0) global_variable(\)\)) bin(&)bin(&) reserved([)reserved([) operator(-)ident(d) string<delimiter(")content(/usr/local/rvm)delimiter(")> reserved(])reserved(])plain(
)  reserved(then) instance_variable(rvm_path)operator(=)string<delimiter(")content(/usr/local/rvm)delimiter(")>plain(
)  reserved(elif) reserved([)reserved([) operator(-)ident(d) string<delimiter(")instance_variable(${HOME})content(/.rvm)delimiter(")> reserved(])reserved(])plain(
)  reserved(then) instance_variable(rvm_path)operator(=)string<delimiter(")instance_variable(${HOME})content(/.rvm)delimiter(")>plain(
)  reserved(elif) reserved([)reserved([) operator(-)ident(d) string<delimiter(")content(/usr/local/rvm)delimiter(")> reserved(])reserved(])plain(
)  reserved(then) instance_variable(rvm_path)operator(=)string<delimiter(")content(/usr/local/rvm)delimiter(")>plain(
)  reserved(else) method(echo) string<delimiter(")content(Can't find rvm install!)delimiter(")> integer(1)bin(>)bin(&)integer(2) delimiter(;) method(exit) integer(1)plain(
)  reserved(fi)plain(
)reserved(fi)plain(
)plain(
)ident(true) instance_variable(${rvm_scripts_path:="$rvm_path/scripts"})plain(
)plain(
)ident(__rvm_shell_lookup_script)operator(()(\)) reserved({)plain(
)  method(local) ident(relative_scripts_dir)plain(
)  instance_variable(relative_scripts_dir)operator(=)string<delimiter(")shell<delimiter($()content(dirname -- )delimiter(")
E
Finished in 1.071453 seconds.

  1) Error:
test_ALL(Bash):
RuntimeError: we are inside [:string, :shell], not string
    /Users/murphy/ruby/coderay/lib/coderay/encoders/debug.rb:44:in `end_group'
    /Users/murphy/ruby/coderay/lib/coderay/encoder.rb:118:in `token'
    /Users/murphy/ruby/coderay/lib/coderay/encoder.rb:186:in `block in compile'
    /Users/murphy/ruby/coderay/lib/coderay/encoder.rb:181:in `each'
    /Users/murphy/ruby/coderay/lib/coderay/encoder.rb:181:in `compile'
    /Users/murphy/ruby/coderay/lib/coderay/encoder.rb:76:in `encode_tokens'
    /Users/murphy/ruby/coderay/test/scanners/coderay_suite.rb:339:in `block in complete_test'
    /Users/murphy/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
    /Users/murphy/ruby/coderay/test/scanners/coderay_suite.rb:338:in `complete_test'
    /Users/murphy/ruby/coderay/test/scanners/coderay_suite.rb:247:in `example_test'
    /Users/murphy/ruby/coderay/test/scanners/coderay_suite.rb:199:in `block (2 levels) in examples_test'
    /Users/murphy/ruby/coderay/test/scanners/coderay_suite.rb:178:in `each'
    /Users/murphy/ruby/coderay/test/scanners/coderay_suite.rb:178:in `block in examples_test'
    /Users/murphy/ruby/coderay/test/scanners/coderay_suite.rb:124:in `block in dir'
    /Users/murphy/ruby/coderay/test/scanners/coderay_suite.rb:123:in `chdir'
    /Users/murphy/ruby/coderay/test/scanners/coderay_suite.rb:123:in `dir'
    /Users/murphy/ruby/coderay/test/scanners/coderay_suite.rb:168:in `examples_test'
    /Users/murphy/ruby/coderay/test/scanners/coderay_suite.rb:157:in `block in test_ALL'
    /Users/murphy/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
    /Users/murphy/ruby/coderay/test/scanners/coderay_suite.rb:154:in `test_ALL'

Can you look into that?

[pejuko]

Im working on this. Unfotunately im without internet and time for next week.

[korny]

Great :-) Thank you.

[funglaub]

Any updates on this issue? Really would like to see Bash being added to CodeRay.

[pejuko]

seems latest coderay_bash (1.0.4) does not break with this file. there are, however, still highlighting bugs

[korny]

I updated the scanner to the last version from coderay_bash. Unfortunately, it's still broken.

A simple check:

> CodeRay.scan('"$("', :bash).debug

string<delimiter(")shell<delimiter($()delimiter(")
CodeRay::Scanners::Scanner::ScanError: 

***ERROR in scanner.rb:200:in `rescue in tokenize': we are inside [:string, :shell], not string (after 0 tokens)

tokens:


current line: 1  column: 5  pos: 4
matched: "\""  state: "Error in CodeRay::Scanners::Bash#scan_tokens, initial state was: :quote"
bol? = false,  eos? = true

surrounding code:
nil  ~~  ""


***ERROR***


    from /Users/murphy/ruby/coderay/lib/coderay/encoders/debug.rb:45:in `end_group'
    from /Users/murphy/ruby/coderay/lib/coderay/scanners/bash.rb:211:in `scan_tokens'
    from /Users/murphy/ruby/coderay/lib/coderay/scanner.rb:197:in `tokenize'
    from /Users/murphy/ruby/coderay/lib/coderay/encoder.rb:85:in `encode'
    from /Users/murphy/ruby/coderay/lib/coderay.rb:198:in `encode'
    from /Users/murphy/ruby/coderay/lib/coderay/tokens_proxy.rb:23:in `encode'
    from /Users/murphy/ruby/coderay/lib/coderay/tokens_proxy.rb:32:in `method_missing'
    from (irb):2
    from /Users/murphy/.rvm/rubies/ruby-2.0.0-p195/bin/irb:16:in `<main>'

The problem is in the way it tries to handle nested strings – it doesn't. Instead of properly using a stack, it only has on/off flags for @quote and @shell.

In the current state, I can't accept this scanner.

Anybody wants to try and fix this one? A working implementation of nested strings can be found in the Ruby scanner (inline_block_stack).

[ghost]

In case of interest, the candidate scanners were removed with a1582d7. As a
workaround, I am now using Rouge, which has a shell/bash lexer already.