Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Videos: ArgumentError: string contains null byte #69

Closed
maia opened this issue Aug 17, 2015 · 46 comments
Closed

Videos: ArgumentError: string contains null byte #69

maia opened this issue Aug 17, 2015 · 46 comments
Labels
bug

Comments

@maia
Copy link
Contributor

maia commented Aug 17, 2015

LinkThumbnailer throws "ArgumentError: string contains null byte“ where querying the same url via VideoInfo causes no problems. See here:

url = "https://www.youtube.com/watch?v=Y45Zq-yKpRw“

video = VideoInfo.new(url); video.title
=> "Robert Mitchum - Julian Cope“

lt = LinkThumbnailer.generate(url); lt.title
ArgumentError: string contains null byte
    from …/gems/ruby-2.2.2/gems/oga-1.2.2/lib/oga/xml/lexer.rb:204:in `advance_native'
    from …/gems/ruby-2.2.2/gems/oga-1.2.2/lib/oga/xml/lexer.rb:204:in `block in advance'
    from …/gems/ruby-2.2.2/gems/oga-1.2.2/lib/oga/xml/lexer.rb:147:in `read_data'
    from …/gems/ruby-2.2.2/gems/oga-1.2.2/lib/oga/xml/lexer.rb:203:in `advance'
    from …/gems/ruby-2.2.2/gems/oga-1.2.2/lib/oga/xml/parser.rb:245:in `each_token'
    from …/gems/ruby-2.2.2/gems/oga-1.2.2/lib/oga/xml/parser.rb:282:in `parse'
    from …/gems/ruby-2.2.2/gems/oga-1.2.2/lib/oga/xml/parser.rb:282:in `parse'
    from …/gems/ruby-2.2.2/gems/oga-1.2.2/lib/oga/oga.rb:27:in `parse_html'
    from …/gems/ruby-2.2.2/gems/video_info-2.5.0/lib/video_info/providers/youtube_scraper.rb:91:in `_set_data_from_api'
    from …/gems/ruby-2.2.2/gems/video_info-2.5.0/lib/video_info/provider.rb:32:in `data'
    from …/gems/ruby-2.2.2/gems/video_info-2.5.0/lib/video_info/providers/youtube_scraper.rb:80:in `available?'
    from …/gems/ruby-2.2.2/gems/video_info-2.5.0/lib/video_info/providers/youtube_scraper.rb:68:in `itemprop_node_value'
    from …/gems/ruby-2.2.2/gems/video_info-2.5.0/lib/video_info/providers/youtube_scraper.rb:18:in `duration'
    from …/gems/ruby-2.2.2/gems/link_thumbnailer-3.0.2/lib/link_thumbnailer/video_parser.rb:27:in `duration'
    from …/gems/ruby-2.2.2/gems/link_thumbnailer-3.0.2/lib/link_thumbnailer/models/video.rb:14:in `initialize'
    from …/gems/ruby-2.2.2/gems/link_thumbnailer-3.0.2/lib/link_thumbnailer/scrapers/opengraph/video.rb:29:in `new'
... 29 levels…
@gottfrois
Copy link
Owner

Seems to be working fine with v3.0.3

@maia
Copy link
Contributor Author

maia commented Aug 24, 2015

Unfortunately I'm still having the same problem:

> url = "https://www.youtube.com/watch?v=Y45Zq-yKpRw"
 => "https://www.youtube.com/watch?v=Y45Zq-yKpRw"
> lt = LinkThumbnailer.generate(url); lt.title
ArgumentError: string contains null byte
    from …/gems/ruby-2.2.2/gems/oga-1.2.3/lib/oga/xml/lexer.rb:204:in `advance_native'
    from …/gems/ruby-2.2.2/gems/oga-1.2.3/lib/oga/xml/lexer.rb:204:in `block in advance'
    from …/gems/ruby-2.2.2/gems/oga-1.2.3/lib/oga/xml/lexer.rb:147:in `read_data'
    from …/gems/ruby-2.2.2/gems/oga-1.2.3/lib/oga/xml/lexer.rb:203:in `advance'
    from …/gems/ruby-2.2.2/gems/oga-1.2.3/lib/oga/xml/parser.rb:245:in `each_token'
    from …/gems/ruby-2.2.2/gems/oga-1.2.3/lib/oga/xml/parser.rb:282:in `parse'
    from …/gems/ruby-2.2.2/gems/oga-1.2.3/lib/oga/xml/parser.rb:282:in `parse'
    from …/gems/ruby-2.2.2/gems/oga-1.2.3/lib/oga/oga.rb:27:in `parse_html'
    from …/gems/ruby-2.2.2/gems/video_info-2.5.0/lib/video_info/providers/youtube_scraper.rb:91:in `_set_data_from_api'
    from …/gems/ruby-2.2.2/gems/video_info-2.5.0/lib/video_info/provider.rb:32:in `data'
    from …/gems/ruby-2.2.2/gems/video_info-2.5.0/lib/video_info/providers/youtube_scraper.rb:80:in `available?'
    from …/gems/ruby-2.2.2/gems/video_info-2.5.0/lib/video_info/providers/youtube_scraper.rb:68:in `itemprop_node_value'
    from …/gems/ruby-2.2.2/gems/video_info-2.5.0/lib/video_info/providers/youtube_scraper.rb:18:in `duration'
    from …/gems/ruby-2.2.2/gems/link_thumbnailer-3.0.3/lib/link_thumbnailer/video_parser.rb:27:in `duration'
    from …/gems/ruby-2.2.2/gems/link_thumbnailer-3.0.3/lib/link_thumbnailer/models/video.rb:14:in `initialize'
    from …/gems/ruby-2.2.2/gems/link_thumbnailer-3.0.3/lib/link_thumbnailer/scrapers/opengraph/video.rb:29:in `new'
... 29 levels...

@0mega
Copy link

0mega commented Aug 25, 2015

I started looking into this, and I'm getting an SSL dependency error (I've also tried it with verify_ssl: false):

2.1.2 :005 > linkt = LinkThumbnailer.generate('https://www.youtube.com/watch?v=liXOb5JXg6w')
Net::HTTP::Persistent::Error: OpenSSL is not available
    from /Users/joanaperrynava/.rvm/gems/ruby-2.1.2/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:597:in `connection_for'
    from /Users/joanaperrynava/.rvm/gems/ruby-2.1.2/gems/net-http-persistent-2.9.4/lib/net/http/persistent.rb:994:in `request'
    from /Users/joanaperrynava/.rvm/gems/ruby-2.1.2/gems/link_thumbnailer-3.0.3/lib/link_thumbnailer/processor.rb:54:in `perform_request'
    from /Users/joanaperrynava/.rvm/gems/ruby-2.1.2/gems/link_thumbnailer-3.0.3/lib/link_thumbnailer/processor.rb:27:in `block in call'
    from /Users/joanaperrynava/.rvm/gems/ruby-2.1.2/gems/link_thumbnailer-3.0.3/lib/link_thumbnailer/processor.rb:37:in `with_valid_url'
    from /Users/joanaperrynava/.rvm/gems/ruby-2.1.2/gems/link_thumbnailer-3.0.3/lib/link_thumbnailer/processor.rb:24:in `call'
    from /Users/joanaperrynava/.rvm/gems/ruby-2.1.2/gems/link_thumbnailer-3.0.3/lib/link_thumbnailer/page.rb:17:in `generate'
    from /Users/joanaperrynava/.rvm/gems/ruby-2.1.2/gems/link_thumbnailer-3.0.3/lib/link_thumbnailer.rb:16:in `generate'
    from (irb):5
    from /Users/joanaperrynava/.rvm/rubies/ruby-2.1.2/bin/irb:11:in `<main>'

Note: I can use OpenSLL and it's already required in irb where I'm running link_thumbnailer

@gottfrois
Copy link
Owner

Yeah, you are not the only one with this issue #67. I should really dig dipper into that SSL issue. Thanks for looking at it

@0mega
Copy link

0mega commented Aug 25, 2015

Sorry, didn't see that issue open. I'll try to help with SSL problem during this week.

@maia
Copy link
Contributor Author

maia commented Aug 26, 2015

gottfrois, do you need any additional feedback regarding the problem with youtube videos? Just asking because the issue still has the "needs feedback" label.

@gottfrois
Copy link
Owner

no thanks @maia I should be able to find what's going on :)

0mega added a commit to 0mega/link_thumbnailer that referenced this issue Aug 27, 2015
@0mega
fix: Ensure Video scraping on the same protocol as original request. F…
592309c 
…ixes gottfrois#69
@0mega 0mega mentioned this issue Aug 27, 2015
Closed
@aryalsan
Copy link

aryalsan commented Sep 2, 2015

I am getting the following error.

from -e:1:in `<main>'irb(main):004:0> linkt = LinkThumbnailer.generate('https://www.youtube.com/watch?v=liXOb5JXg6w')
ETHON: started MULTI
ETHON:         performed EASY effective_url=https://i.ytimg.com/vi/liXOb5JXg6w/hqdefault.jpg response_code=200 return_code=write_error total_time=0.878789
ETHON: performed MULTI
ArgumentError: string contains null byte

@0mega 0mega mentioned this issue Sep 2, 2015
Closed
@0mega
Copy link

0mega commented Sep 2, 2015

@aryalsan you can use my PR as a workaround while waiting for a fix in video_info gem or maybe even Kernel.open method.

@0mega 0mega mentioned this issue Sep 3, 2015
Closed
@aryalsan
Copy link

Any updates coming recently!

@gottfrois
Copy link
Owner

I'm waiting for video_info gem to release their master into a new version :)

@kurtish
Copy link

kurtish commented Sep 16, 2015

We are having the same problems and not of the links to you tube videos are able to generate images. The weird part is that your thumbnailer demo works fine for your tube, but it is no longer working for us.

@gottfrois
Copy link
Owner

The demo is using this version of LinkThumbnailer https://github.com/gottfrois/link_thumbnailer_api/blob/master/Gemfile.lock#L59

@kurtish
Copy link

kurtish commented Sep 16, 2015

@gottfrois Thanks, is it running on heroku on a Cedar 14 standard build as well? Just wanted to check as to know if it is the heroku base version of imagemagick being used as well.

@gottfrois
Copy link
Owner

yes it's a standard heroku, nothing fancy. However there are no imagemagick dependency. Where do you see it?

@aryalsan
Copy link

Still waiting for the updates! Any news yet!!!

@0mega
Copy link

0mega commented Sep 25, 2015

@aryalsan this won't be done until this issue is solved so you are welcome to help.

@idjuradj
Copy link

Newbie here, with the same error as above.

Is there a simple way to make this to work?

I see the demo version is working and it's using v3.0.2? Should I downgrade?

@0mega, you mentioned a workaround, can you provide more details about this? I changed the dependency to 'video_info', '~> 2.5', but it's still not solving my problem?

@aaronsakowski
Copy link

Still testing, but adding this to the Gemfile works so far for me.

gem 'video_info', github: 'thibaudgg/video_info'

Found here: thibaudgg/video_info#100

@0mega
Copy link

0mega commented Oct 1, 2015

Hi @idjuradj, you can use my workaround 0mega@592309c, or just rebuild link_thumbnailer locally, pointing to video_info gem master as @aaronsakowski mentioned.

@gottfrois
Copy link
Owner

I'll create a branch that have gemfile points to video_info master branch for you guys

@gottfrois
Copy link
Owner

Use this branch guys, until they release master in a new video_info version https://github.com/gottfrois/link_thumbnailer/tree/tmp-video-info-fix

@aryalsan
Copy link

Cool, it works for the Youtube video but for Vimeo video, its generates unexpected network error while fetching information about the video.

I think its the problem of the Video_info gem

@0mega
Copy link

0mega commented Oct 11, 2015

@aryalsan I didn't look into it but it's probably because of the API token that is required to fetch information about the video from vimeo. In version 2.6 of video_info, there will be a vimeo scraper so you won't need an API key.
This does not happen with youtube videos because there is already an implementation of a youtube scraper.

@maia
Copy link
Contributor Author

maia commented Oct 18, 2015

When using the tmp-video-info-fix branch (and adding gem 'video_info', github: 'thibaudgg/video_info' to Gemfile) parsing youtube videos works.

Parsing a vimeo.com video generates a "VideoInfo::HttpError: unexpected network error while fetching information about the video" and a "OpenURI::HTTPError: 401 Authorization Required".

I assume we need to wait for video_info 2.6, @0mega do you have a time frame? Thanks!

@vheuken
Copy link

vheuken commented Oct 27, 2015

@maia There's just one thing left before the 2.6 release. Should be done within 1-2 weeks.

@gottfrois
Copy link
Owner

👍

1 similar comment
@aryalsan
Copy link

👍

@aryalsan
Copy link

@vheuken is there any possible dates of the release of Video info gem 2.6

@vheuken
Copy link

vheuken commented Nov 29, 2015

@aryalsan Sorry for the delay. I've been busier than I've expected so I haven't been able to work on it. I'll make some time for it tonight and I should be able to finish it up.

@aryalsan
Copy link

@vheuken, cool, and we are eagerly waiting for the 2.6 update release. And also would you please check for the Facebook videos, i think its generating the wrong thumbnail URL.

@vheuken
Copy link

vheuken commented Nov 30, 2015

Just implemented most of the last task before 2.6 release. Here's what's left: thibaudgg/video_info#95 (comment)

Should be done within a few days.

And also would you please check for the Facebook videos, i think its generating the wrong thumbnail URL.

video_info doesn't handle Facebook videos

@maia maia mentioned this issue Jan 10, 2016
Closed
@gottfrois gottfrois mentioned this issue Jan 19, 2016
Closed
@gottfrois
Copy link
Owner

What's up on this @vheuken ? And happy new year man :)

@vheuken
Copy link

vheuken commented Jan 19, 2016

Sorry, I still have to finish the Vimeo Playlist scraper. If you want, you can try using the branch with the scraper implemented in your project.

gem 'video_info', :git => 'https://github.com/thibaudgg/video_info', :branch => 'vimeo-scraper'

@vheuken
Copy link

vheuken commented Mar 1, 2016

Can you guys try the master branch now? Vimeo scraper should be in and we'll push a new version shortly. Sorry for the wait!

@vheuken
Copy link

vheuken commented Mar 2, 2016

video_info 2.6 has been released with the Vimeo scraper. Hopefully this resolves your issue!

@gottfrois
Copy link
Owner

👍 I will try that when I have a moment :) Thanks for the notice

@aryalsan
Copy link

aryalsan commented Mar 2, 2016

Thanks for the release.
On Wed, Mar 2, 2016 at 8:18 PM Pierre-Louis Gottfrois <
[email protected]> wrote:

[image: 👍] I will try that when I have a moment :) Thanks for the
notice

—
Reply to this email directly or view it on GitHub
#69 (comment)
.

@maia
Copy link
Contributor Author

maia commented Mar 5, 2016

The issue is fixed with video_info 2.6.0 (thanks, @vheuken !). @gottfrois can you push a new version of link_thumbnailer? (plus, there are a few forks that look like they could be pulled in). Thanks everyone!

@gottfrois
Copy link
Owner

Released in v3.1.1. Thanks every one to make it happen :)

@vheuken
Copy link

vheuken commented Mar 5, 2016

Please keep in mind that, due to the nature of scrapers, future breakage is a very real possibility. If you ever have an issue I'll be more than happy to release a patch fix.

@gottfrois
Copy link
Owner

Of course, thanks for the reminder. What are our options to limit them? Using each provider APIs directly I imagine?

@vheuken
Copy link

vheuken commented Mar 7, 2016

I think you should point out to users that they are able to set a Vimeo API key as follows:

VideoInfo.provider_api_keys = { vimeo: 'YOUR_VIMEO_API_KEY' }

(Of course, you are free to make your own interface around that).

@gottfrois
Copy link
Owner

Right, I'll probably include that in the gem config

@vheuken
Copy link

vheuken commented Mar 8, 2016

Just a heads up, we're getting ready for a 2.7 release in the coming weeks where we significantly reduced our gem dependencies, so it will probably be a good idea to update to that when we release that. I'll try to ping you or send over a PR when it's ready. I'm sure your users will appreciate having to pull in a few fewer gems.

@gottfrois
Copy link
Owner

@vheuken sounds great :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@maia @0mega @vheuken @gottfrois @aaronsakowski @kurtish @aryalsan @idjuradj