ruby on rails - Curl:: Err:: PartialFileError: transferido un error de archivo parcial

ruby-on-rails

Question

Estoy tratando de ejecutar el siguiente código en la consola de rieles:

ce = Curl::Easy.new("http://www.homestolove.com.au/bathroom-profile-fresh-approach-2391")
ce.verbose=true
ce.perform

Pero estoy recibiendo un error de seguimiento. ¿Alguien puede sugerir cómo solucionar esto? Parece que esta url es bastante lenta al devolver la respuesta. ¿Podemos hacer algo para evitar que la conexión termine hasta que se complete la transferencia de archivos?

* Adding handle: conn: 0x95f3210
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x95f3210) send_pipe: 1, recv_pipe: 0
* About to connect() to www.homestolove.com.au port 80 (#0)
* Trying 52.85.77.138...
* Connected to www.homestolove.com.au (52.85.77.138) port 80 (#0)
> GET /bathroom-profile-fresh-approach-2391 HTTP/1.1
Host: www.homestolove.com.au
Accept: */*
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
< Content-Length: 717180
< Cache-Control: public, max-age=1800
< Cache-Control: no-cache="set-cookie"
< Date: Sat, 29 Oct 2016 06:22:49 GMT
< ETag: W/"af17c-wej92PccZYKK1ePeKXoQ1Q"
* Server nginx is not blacklisted
< Server: nginx
< Strict-Transport-Security: max-age=31536000; includeSubdomains
< X-Frame-Options: SAMEORIGIN
< X-Powered-By: Express
< X-XSS-Protection: 1; mode=block
< Vary: Accept-Encoding,Accept-Encoding
< X-Cache: Miss from cloudfront
< Via: 1.1 94812f867206232855f09570ec52b557.cloudfront.net (CloudFront), 1.1 wsg.Lehi05
< X-Amz-Cf-Id: qiwFsn3d4lSmNZTSC_BVojcZmWjU1Toba7XagEiFCe-zlywo6GlkVw==
< Age: 2
<
* transfer closed with 431831 bytes remaining to read
* Closing connection 0
Curl::Err::PartialFileError: Transferred a partial file
from /opt/rails/test/shared/bundle/ruby/1.9.1/gems/curb-0.9.3/lib/curl/easy.rb:73:in `perform''
from (irb):5
from /opt/rails/test/shared/bundle/ruby/1.9.1/bundler/gems/rails-efa67857350e/railties/lib/rails/commands/console.rb:47:in `start''
from /opt/rails/test/shared/bundle/ruby/1.9.1/bundler/gems/rails-efa67857350e/railties/lib/rails/commands/console.rb:8:in `start''
from /opt/rails/test/shared/bundle/ruby/1.9.1/bundler/gems/rails-efa67857350e/railties/lib/rails/commands.rb:41:in `<top (required)>''
from script/rails:6:in `require''
from script/rails:6:in `<main>''

Answer 1

El sitio tiene un encabezado "Content-Length" incorrecto. Por lo tanto, Curl no puede analizarlo correctamente.

Así que saltémonos el encabezado, y dejemos que curl lo haga solo

ce = Curl::Easy.new("http://www.homestolove.com.au/bathroom-profile-fresh-approach-2391")
ce.verbose = true
ce.ignore_content_length = true # add the line
ce.perform

Y el resultado está aquí:

ce.body

Espero que te haya ayudado

Answer 2

Parece que funciona con la encoding establecida en gzip

ce = Curl::Easy.new("http://www.homestolove.com.au/bathroom-profile-fresh-approach-2391")
ce.verbose = true
ce.encoding = ''gzip''
ce.perform

salida:

* Trying 216.137.61.96...
* Connected to www.homestolove.com.au (216.137.61.96) port 80 (#0)
> GET /bathroom-profile-fresh-approach-2391 HTTP/1.1
Host: www.homestolove.com.au
Accept: */*
Accept-Encoding: gzip
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Cache-Control: public, max-age=1800
< Cache-Control: no-cache="set-cookie"
< Content-Encoding: gzip
< Date: Sun, 06 Nov 2016 19:49:27 GMT
< ETag: W/"a7c31-QuyLkXEk3HhTu/KMM5QDNQ"
< Server: nginx
< Strict-Transport-Security: max-age=31536000; includeSubdomains
< X-Frame-Options: SAMEORIGIN
< X-Powered-By: Express
< X-XSS-Protection: 1; mode=block
< Vary: Accept-Encoding
< Age: 162
< X-Cache: Hit from cloudfront
< Via: 1.1 e24fef4a7b03bd84e1e8d57f2471a84d.cloudfront.net (CloudFront)
< X-Amz-Cf-Id: 96AZ5p0ts5UZdPXTEpCk8S3wEQJk9a4vZD7vhY-8sXHX_NTyGSUouA==
<