C Extension project - Async HTTP transfer - CURL?

thomthom · 26 Oct 2011, 07:51

Ok - so I want to get a solution for transferring binary files in the background.

Initial quick search presented CURL - and I'm looking at some of the examples, progress meter and threaded. Would someone with some better C experience have a quick look and tell if merging these two examples would allow us to have a Ruby C Extension that can download files in the background with progress callback functionality?

http://curl.haxx.se/libcurl/c/progressfunc.html
http://curl.haxx.se/libcurl/c/multithread.html

Dan Rathbun · 26 Oct 2011, 08:10

Dana's done some of the work I think:
https://github.com/danawoodman/google-sketchup-file-downloader

thomthom · 26 Oct 2011, 08:22

Ah - awesome! (I think I should have known this... )

I'll jump on that project. I see the todo list mentioneds it needs some SketchUp and corss platform testing.

draftomatic · 8 Nov 2011, 01:04

So how does this run in the background? I thought threading wasn't possible in SketchUp, even using an extension?

thomthom · 8 Nov 2011, 07:09

I don't know how that projects works yet. I've not looked at it properly.

But surely C threads would be possible? I thought it was only Ruby threads that was a no-go...

AdamB · 8 Nov 2011, 10:02

Ruby has "green threads" which is simply the Ruby interpreter locally slicing & dicing its own execution thread. But since the Ruby intepreter itself is being scheduled from the runloop of SketchUp, its of little use for asyncronous work.

However, you can create as many real Processes / Threads / Fibres as you like using C - but you can't talk to the SketchUp API with them because its not designed for this (not thread-safe). You can only make calls into SketchUp from the execution thread that is called from Sketchup.

So kicking off a process that does a blocking read on a socket is fine. But you'll have to have a protocol for getting the data back into the scope of your C Extension - probably by periodically polling to see if the worker thread is done.

For my money, integrating libCURL seems nuts. Just spawn a shell command to run wget or curl and stick it in a file somewhere.

draftomatic · 8 Nov 2011, 21:45

@adamb said:

However, you can create as many real Processes / Threads / Fibres as you like using C - but you can't talk to the SketchUp API with them because its not designed for this (not thread-safe). You can only make calls into SketchUp from the execution thread that is called from Sketchup.

Spawning a subprocess is not the same thing as running a Ruby C extension.

draftomatic · 9 Nov 2011, 03:08

So can you use C extensions to run asynchronous tasks?

thomthom · 9 Nov 2011, 13:38

@adamb said:

So kicking off a process that does a blocking read on a socket is fine. But you'll have to have a protocol for getting the data back into the scope of your C Extension - probably by periodically polling to see if the worker thread is done.

Can the polling also get more info? Say get an variable indicating progress?

tomasz · 9 Nov 2011, 13:45

@adamb said:

Ruby has "green threads" which is simply the Ruby interpreter locally slicing & dicing its own execution thread. But since the Ruby intepreter itself is being scheduled from the runloop of SketchUp, its of little use for asyncronous work.

In other words, if I have a progress bar inside a C extension, it will suffer same 'halt' as whole SU user interface, when Ruby is doing heavy lifting, right?
Does it mean that I have to start an independent thread with the progress bar to have it updated in same conditions?