Ripper is a Ruby script parser.
You can get information from the parser with event-based style. Information such as abstract syntax trees or simple lexical analysis of the Ruby program.
Ripper provides an easy interface for parsing your program into a symbolic expression tree (or S-expression).
Understanding the output of the parser may come as a challenge, it's recommended you use PP to format the output for legibility.
require 'ripper' require 'pp' pp Ripper.sexp('def hello(world) "Hello, #{world}!"; end') #=> [:program, [[:def, [:@ident, "hello", [1, 4]], [:paren, [:params, [[:@ident, "world", [1, 10]]], nil, nil, nil, nil, nil, nil]], [:bodystmt, [[:string_literal, [:string_content, [:@tstring_content, "Hello, ", [1, 18]], [:string_embexpr, [[:var_ref, [:@ident, "world", [1, 27]]]]], [:@tstring_content, "!", [1, 33]]]]], nil, nil, nil]]]]
You can see in the example above, the expression starts with :program
.
From here, a method definition at :def
, followed by the method's identifier :@ident
. After the method's identifier comes the parentheses :paren
and the method parameters under :params
.
Next is the method body, starting at :bodystmt
(stmt
meaning statement), which contains the full definition of the method.
In our case, we're simply returning a String, so next we have the :string_literal
expression.
Within our :string_literal
you'll notice two @tstring_content
, this is the literal part for Hello,
and !
. Between the two @tstring_content
statements is a :string_embexpr
, where embexpr is an embedded expression. Our expression consists of a local variable, or var_ref
, with the identifier (@ident
) of world
.
ruby 1.9 (support CVS HEAD only)
bison 1.28 or later (Other yaccs do not work)
Ruby License. Minero Aoki [email protected] http://i.loveruby.net
This array contains name of all ripper events.
This array contains name of parser events.
This array contains name of scanner events.
version of Ripper
Tokenizes the Ruby program and returns an array of an array, which is formatted like [[lineno, column], type, token]
.
require 'ripper' require 'pp' pp Ripper.lex("def m(a) nil end") #=> [[[1, 0], :on_kw, "def"], [[1, 3], :on_sp, " " ], [[1, 4], :on_ident, "m" ], [[1, 5], :on_lparen, "(" ], [[1, 6], :on_ident, "a" ], [[1, 7], :on_rparen, ")" ], [[1, 8], :on_sp, " " ], [[1, 9], :on_kw, "nil"], [[1, 12], :on_sp, " " ], [[1, 13], :on_kw, "end"]]
# File ext/ripper/lib/ripper/lexer.rb, line 42 def Ripper.lex(src, filename = '-', lineno = 1) Lexer.new(src, filename, lineno).lex end
Create a new Ripper object. src must be a String, an IO, or an Object which has gets method.
This method does not starts parsing. See also #parse and ::parse.
static VALUE ripper_initialize(int argc, VALUE *argv, VALUE self) { struct parser_params *parser; VALUE src, fname, lineno; TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser); rb_scan_args(argc, argv, "12", &src, &fname, &lineno); if (RB_TYPE_P(src, T_FILE)) { parser->parser_lex_gets = ripper_lex_get_generic; } else { StringValue(src); parser->parser_lex_gets = lex_get_str; } parser->parser_lex_input = src; parser->eofp = Qfalse; if (NIL_P(fname)) { fname = STR_NEW2("(ripper)"); OBJ_FREEZE(fname); } else { StringValue(fname); fname = rb_str_new_frozen(fname); } parser_initialize(parser); parser->parser_ruby_sourcefile_string = fname; parser->parser_ruby_sourcefile = RSTRING_PTR(fname); parser->parser_ruby_sourceline = NIL_P(lineno) ? 0 : NUM2INT(lineno) - 1; return Qnil; }
Parses src
and create S-exp tree. Returns more readable tree rather than ::sexp_raw. This method is mainly for developer use.
require 'ripper' require 'pp' pp Ripper.sexp("def m(a) nil end") #=> [:program, [[:def, [:@ident, "m", [1, 4]], [:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil, nil]], [:bodystmt, [[:var_ref, [:@kw, "nil", [1, 9]]]], nil, nil, nil]]]]
# File ext/ripper/lib/ripper/sexp.rb, line 30 def Ripper.sexp(src, filename = '-', lineno = 1) builder = SexpBuilderPP.new(src, filename, lineno) sexp = builder.parse sexp unless builder.error? end
Parses src
and create S-exp tree. This method is mainly for developer use.
require 'ripper' require 'pp' pp Ripper.sexp_raw("def m(a) nil end") #=> [:program, [:stmts_add, [:stmts_new], [:def, [:@ident, "m", [1, 4]], [:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil]], [:bodystmt, [:stmts_add, [:stmts_new], [:var_ref, [:@kw, "nil", [1, 9]]]], nil, nil, nil]]]]
# File ext/ripper/lib/ripper/sexp.rb, line 56 def Ripper.sexp_raw(src, filename = '-', lineno = 1) builder = SexpBuilder.new(src, filename, lineno) sexp = builder.parse sexp unless builder.error? end
Parses src
and return a string which was matched to pattern
. pattern
should be described as Regexp.
require 'ripper' p Ripper.slice('def m(a) nil end', 'ident') #=> "m" p Ripper.slice('def m(a) nil end', '[ident lparen rparen]+') #=> "m(a)" p Ripper.slice("<<EOS\nstring\nEOS", 'heredoc_beg nl $(tstring_content*) heredoc_end', 1) #=> "string\n"
# File ext/ripper/lib/ripper/lexer.rb, line 84 def Ripper.slice(src, pattern, n = 0) if m = token_match(src, pattern) then m.string(n) else nil end end
Tokenizes the Ruby program and returns an array of strings.
p Ripper.tokenize("def m(a) nil end") # => ["def", " ", "m", "(", "a", ")", " ", "nil", " ", "end"]
# File ext/ripper/lib/ripper/lexer.rb, line 20 def Ripper.tokenize(src, filename = '-', lineno = 1) Lexer.new(src, filename, lineno).tokenize end
Return column number of current parsing line. This number starts from 0.
static VALUE ripper_column(VALUE self) { struct parser_params *parser; long col; TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser); if (!ripper_initialized_p(parser)) { rb_raise(rb_eArgError, "method called for uninitialized object"); } if (NIL_P(parser->parsing_thread)) return Qnil; col = parser->tokp - parser->parser_lex_pbeg; return LONG2NUM(col); }
Return encoding of the source.
VALUE rb_parser_encoding(VALUE vparser) { struct parser_params *parser; TypedData_Get_Struct(vparser, struct parser_params, &parser_data_type, parser); return rb_enc_from_encoding(current_enc); }
Return true if parsed source ended by +_END_+.
VALUE rb_parser_end_seen_p(VALUE vparser) { struct parser_params *parser; TypedData_Get_Struct(vparser, struct parser_params, &parser_data_type, parser); return ruby__end__seen ? Qtrue : Qfalse; }
Return true if parsed source has errors.
static VALUE ripper_error_p(VALUE vparser) { struct parser_params *parser; TypedData_Get_Struct(vparser, struct parser_params, &parser_data_type, parser); return parser->error_p ? Qtrue : Qfalse; }
Return current parsing filename.
static VALUE ripper_filename(VALUE self) { struct parser_params *parser; TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser); if (!ripper_initialized_p(parser)) { rb_raise(rb_eArgError, "method called for uninitialized object"); } return parser->parser_ruby_sourcefile_string; }
Return line number of current parsing line. This number starts from 1.
static VALUE ripper_lineno(VALUE self) { struct parser_params *parser; TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser); if (!ripper_initialized_p(parser)) { rb_raise(rb_eArgError, "method called for uninitialized object"); } if (NIL_P(parser->parsing_thread)) return Qnil; return INT2NUM(parser->parser_ruby_sourceline); }
Start parsing and returns the value of the root action.
static VALUE ripper_parse(VALUE self) { struct parser_params *parser; TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser); if (!ripper_initialized_p(parser)) { rb_raise(rb_eArgError, "method called for uninitialized object"); } if (!NIL_P(parser->parsing_thread)) { if (parser->parsing_thread == rb_thread_current()) rb_raise(rb_eArgError, "Ripper#parse is not reentrant"); else rb_raise(rb_eArgError, "Ripper#parse is not multithread-safe"); } parser->parsing_thread = rb_thread_current(); rb_ensure(ripper_parse0, self, ripper_ensure, self); return parser->result; }
Get yydebug.
VALUE rb_parser_get_yydebug(VALUE self) { struct parser_params *parser; TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser); return yydebug ? Qtrue : Qfalse; }
Set yydebug.
VALUE rb_parser_set_yydebug(VALUE self, VALUE flag) { struct parser_params *parser; TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser); yydebug = RTEST(flag); return flag; }
This method is called when the parser found syntax error.
# File ext/ripper/lib/ripper/core.rb, line 55 def compile_error(msg) end
This method is called when weak warning is produced by the parser. fmt
and args
is printf style.
# File ext/ripper/lib/ripper/core.rb, line 46 def warn(fmt, *args) end
This method is called when strong warning is produced by the parser. fmt
and args
is printf style.
# File ext/ripper/lib/ripper/core.rb, line 51 def warning(fmt, *args) end
Ruby Core © 1993–2016 Yukihiro Matsumoto
Licensed under the Ruby License.
Ruby Standard Library © contributors
Licensed under their own licenses.