ruby on rails - shallow - Rails ArgumentError: invalid-encoding no válido

ruby on rails tutorial

Question

Durante el último mes, hemos tenido un bot rastreando nuestro sitio regularmente, lo que ha provocado un montón de errores ArgumentError: invalid %-encoding porque las URL están mal formadas. He visto un montón de problemas en rack here y here y here , y he visto este hilo SO, pero no parece haber una solución definitiva. ¿Hay una solución correcta para los errores GET? ¿Tengo que hacer un parche de parche?

editar: Y aquí hay una traza inversa:

/usr/local/lib/ruby/1.9.1/uri/common.rb:898:in `decode_www_form_component''
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/utils.rb:41:in `unescape''
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/utils.rb:94:in `block (2 levels) in parse_nested_query''
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/utils.rb:94:in `map''
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/utils.rb:94:in `block in parse_nested_query''
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/utils.rb:93:in `each''
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/utils.rb:93:in `parse_nested_query''
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/request.rb:332:in `parse_query''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/http/request.rb:269:in `parse_query''
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/request.rb:186:in `GET''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/http/request.rb:225:in `GET''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/http/parameters.rb:10:in `parameters''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/http/filter_parameters.rb:33:in `filtered_parameters''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_controller/metal/instrumentation.rb:21:in `process_action''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_controller/metal/params_wrapper.rb:207:in `process_action''
[GEM_ROOT]/gems/activerecord-3.2.12/lib/active_record/railties/controller_runtime.rb:18:in `process_action''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/abstract_controller/base.rb:121:in `process''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/abstract_controller/rendering.rb:45:in `process''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_controller/metal.rb:203:in `dispatch''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_controller/metal.rb:246:in `block in action''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:73:in `call''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:73:in `dispatch''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:36:in `call''
[GEM_ROOT]/gems/journey-1.0.4/lib/journey/router.rb:68:in `block in call''
[GEM_ROOT]/gems/journey-1.0.4/lib/journey/router.rb:56:in `each''
[GEM_ROOT]/gems/journey-1.0.4/lib/journey/router.rb:56:in `call''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/routing/route_set.rb:601:in `call''
[GEM_ROOT]/gems/omniauth-1.1.1/lib/omniauth/strategy.rb:177:in `call!''
[GEM_ROOT]/gems/omniauth-1.1.1/lib/omniauth/strategy.rb:157:in `call''
[GEM_ROOT]/gems/sass-3.2.7/lib/sass/plugin/rack.rb:54:in `call''
[GEM_ROOT]/gems/warden-1.2.1/lib/warden/manager.rb:35:in `block in call''
[GEM_ROOT]/gems/warden-1.2.1/lib/warden/manager.rb:34:in `catch''
[GEM_ROOT]/gems/warden-1.2.1/lib/warden/manager.rb:34:in `call''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call''
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/etag.rb:23:in `call''
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/conditionalget.rb:25:in `call''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/head.rb:14:in `call''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/params_parser.rb:21:in `call''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/flash.rb:242:in `call''
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:210:in `context''
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:205:in `call''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/cookies.rb:341:in `call''
[GEM_ROOT]/gems/activerecord-3.2.12/lib/active_record/query_cache.rb:64:in `call''
[GEM_ROOT]/gems/activerecord-3.2.12/lib/active_record/connection_adapters/abstract/connection_pool.rb:479:in `call''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call''
[GEM_ROOT]/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:405:in `_run__497203393471184793__call__4495106819278994598__callbacks''
[GEM_ROOT]/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:405:in `__run_callback''
[GEM_ROOT]/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:385:in `_run_call_callbacks''
[GEM_ROOT]/gems/activesupport-3.2.12/lib/active_support/callbacks.rb:81:in `run_callbacks''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/callbacks.rb:27:in `call''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/remote_ip.rb:31:in `call''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/show_exceptions.rb:56:in `call''
[GEM_ROOT]/gems/railties-3.2.12/lib/rails/rack/logger.rb:32:in `call_app''
[GEM_ROOT]/gems/railties-3.2.12/lib/rails/rack/logger.rb:16:in `block in call''
[GEM_ROOT]/gems/activesupport-3.2.12/lib/active_support/tagged_logging.rb:22:in `tagged''
[GEM_ROOT]/gems/railties-3.2.12/lib/rails/rack/logger.rb:16:in `call''
[GEM_ROOT]/gems/actionpack-3.2.12/lib/action_dispatch/middleware/request_id.rb:22:in `call''
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/methodoverride.rb:21:in `call''
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/runtime.rb:17:in `call''
[GEM_ROOT]/gems/activesupport-3.2.12/lib/active_support/cache/strategy/local_cache.rb:72:in `call''
[GEM_ROOT]/gems/rack-1.4.5/lib/rack/lock.rb:15:in `call''
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:136:in `forward''
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:143:in `pass''
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:172:in `rescue in lookup''
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:168:in `lookup''
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:66:in `call!''
[GEM_ROOT]/gems/rack-cache-1.2/lib/rack/cache/context.rb:51:in `call''
[GEM_ROOT]/gems/railties-3.2.12/lib/rails/engine.rb:479:in `call''
[GEM_ROOT]/gems/railties-3.2.12/lib/rails/application.rb:223:in `call''
[GEM_ROOT]/gems/railties-3.2.12/lib/rails/railtie/configurable.rb:30:in `method_missing''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/rack/request_handler.rb:96:in `process_request''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_request_handler.rb:516:in `accept_and_process_next_request''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_request_handler.rb:274:in `main_loop''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/rack/application_spawner.rb:206:in `start_request_handler''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/rack/application_spawner.rb:171:in `block in handle_spawn_application''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/utils.rb:479:in `safe_fork''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/rack/application_spawner.rb:166:in `handle_spawn_application''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server.rb:180:in `start''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/rack/application_spawner.rb:129:in `start''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/spawn_manager.rb:253:in `block (2 levels) in spawn_rack_application''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server_collection.rb:132:in `lookup_or_add''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/spawn_manager.rb:246:in `block in spawn_rack_application''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server_collection.rb:82:in `block in synchronize''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server_collection.rb:79:in `synchronize''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/spawn_manager.rb:244:in `spawn_rack_application''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/spawn_manager.rb:137:in `spawn_application''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/spawn_manager.rb:275:in `handle_spawn_application''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server.rb:357:in `server_main_loop''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/lib/phusion_passenger/abstract_server.rb:206:in `start_synchronously''
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-3.0.13/helper-scripts/passenger-spawn-server:99:in `<main>''

Answer 1

Podría inyectar un middleware diseñado para detectar estos y fallar con gracia. La idea básica es simplemente intentar analizar la cadena de consulta, y si falla, rescatar con un HTTP 400. De lo contrario, simplemente permita que pase la solicitud.

class RefuseInvalidRequest
  def initialize(app)
    @app = app
  end

  def call(env)
    query = Rack::Utils.parse_nested_query(env[''QUERY_STRING''].to_s) rescue :bad_query
    if query == :bad_query
      [400, {''Content-Type'' => ''text/plain''}, "Bad Request"]
    else
      @app.call(env)
    end
  end
end

No lo he probado, pero el concepto debería funcionar.

Answer 2

si no te molesta que monkeypatching Rack crees en el archivo config/initializers (por ejemplo, rack.rb ) con este contenido:

module Rack
  module Utils
    if defined?(::Encoding)
      def unescape(s, encoding = Encoding::UTF_8)
        begin
          URI.decode_www_form_component(s, encoding)
        rescue ArgumentError
          URI.decode_www_form_component(URI.encode(s), encoding)
        end
      end
    else
      def unescape(s, encoding = nil)
        begin
          URI.decode_www_form_component(s, encoding)
        rescue ArgumentError
          URI.decode_www_form_component(URI.encode(s), encoding)
        end
      end
    end
    module_function :unescape
  end
end

Si funciona con pasajeros, pero con Webrick y Thin no. Parece que tanto webrick como thin analizan una solicitud, por lo que la falla ocurre antes de que se cargue el inicializador. Por ejemplo, con Thin error ocurre en thin-1.6.2/lib/thin/request.rb:84 .