[mobile] Download and parse/scrape web page

Get help using Construct 2

Post » Sat Feb 21, 2015 11:05 am

Hi all

How would I go about downloading a web page's content for analysis/scraping/parsing, in a mobile context (Android/iOS).

Thanks!
B
24
S
9
G
4
Posts: 1,646
Reputation: 6,596

Post » Sat Feb 21, 2015 12:58 pm

Use ajax to request the page, then you will have it in Ajax.LastData.
Who dares wins
B
56
S
16
G
21
Posts: 1,875
Reputation: 19,406

Post » Sat Feb 21, 2015 1:16 pm

lennaert wrote:Use ajax to request the page, then you will have it in Ajax.LastData.


Dang.. I've only used it to get project files. Thanks.

edit: actually I don't seem to get anything in AJAX.LastData if I request a random web page. Does this work only if I target a specific file?
B
24
S
9
G
4
Posts: 1,646
Reputation: 6,596

Post » Sun Feb 22, 2015 11:03 pm

So what should I expect from this code?
You do not have the required permissions to view the files attached to this post.
B
24
S
9
G
4
Posts: 1,646
Reputation: 6,596

Post » Mon Feb 23, 2015 11:54 am

Cant open the capx, I do not have the spritefont+ installed.

But I imagine you have the ajax plugin doing a request or post and its not working.
Hitting Ctrl shift J gives you the error console in chrome and FF.
I forgot about the cross domain policies.
Sites generally have cross domain security enabled. (origin access)

So the ajax is not the best solution unless your trying to fetch info from one of your own sites on the same domain or sites having all origin access set to *

You could use a hidden Iframe, load the page, and fetch the html from the frame with a bit of javascript.
Who dares wins
B
56
S
16
G
21
Posts: 1,875
Reputation: 19,406

Post » Mon Feb 23, 2015 9:08 pm

lennaert wrote:Cant open the capx, I do not have the spritefont+ installed.

But I imagine you have the ajax plugin doing a request or post and its not working.
Hitting Ctrl shift J gives you the error console in chrome and FF.
I forgot about the cross domain policies.
Sites generally have cross domain security enabled. (origin access)

So the ajax is not the best solution unless your trying to fetch info from one of your own sites on the same domain or sites having all origin access set to *

You could use a hidden Iframe, load the page, and fetch the html from the frame with a bit of javascript.


yeah sorry I use SF+ by default.

Yes it's an Ajax request and Ajax.lastData comes up empty. You're right, I checked the Chrome console and it says "No 'Access-Control-Allow-Origin' header is present on the requested resource."

Sorry to be clueless but could you give a bit more info on how I would go about the iFrame technique?
B
24
S
9
G
4
Posts: 1,646
Reputation: 6,596

Post » Tue Feb 24, 2015 12:13 pm

codah wrote:
yeah sorry I use SF+ by default.

Yes it's an Ajax request and Ajax.lastData comes up empty. You're right, I checked the Chrome console and it says "No 'Access-Control-Allow-Origin' header is present on the requested resource."

Sorry to be clueless but could you give a bit more info on how I would go about the iFrame technique?



Sorry to say, I just did some research on the cross site policies, and the methods used a couple years ago now also no longer seem to work.
Well, some still work, in very old browsers :
The current support for the adding of manual scripting in cs2 is rather limited.
I added podes iframe plugin and did some tests, but did not succeed.

There are some other approaches, such as using a webserver with php and having that request a webpage, which relays the results back to you. But its a messy solution.

Outside of construct 2 its already a lot easier as you could add javascript to the iframe, and have the iframe send the innerHTML to the parent object (parent window). But in cs2 I can not add javascript or jquery to podes iframe. Perhaps you can ask Pode if it is possible to have it add it to the plugin so the iframe source is available in the iframes propperties in cs2.

For various approaches you can google around for : javascript get iframe html

I applied several of the suggestions by adding the browser object and podes iframe object.
The iframe loaded a url, and with the browser object I executed various bits of javascript to get the innerHTML from the iframe to anything in cs2, and all that worked gives me the cross domain notifcation.

Code: Select all
document.getElementById('my_iframe').contentWindow.document.body.innerHTML;


:?
Who dares wins
B
56
S
16
G
21
Posts: 1,875
Reputation: 19,406

Post » Tue Feb 24, 2015 9:38 pm

I appreciate all your effort.. looks like I'm out of luck. I just wanted to grab some weather information and display the temperature :/ Do you know of any way to FTP a file into a C2 app?
B
24
S
9
G
4
Posts: 1,646
Reputation: 6,596

Post » Tue Feb 24, 2015 10:01 pm

Not really, I have not been using ftp options in cs2.

Doesnt Rex have anything :D ?


Perhaps the option to get project files ?, and externally placing a file in the app folder ?
Who dares wins
B
56
S
16
G
21
Posts: 1,875
Reputation: 19,406

Post » Tue Feb 24, 2015 10:04 pm

I did scan the add-ons but I'll have another look. Thanks again :D
B
24
S
9
G
4
Posts: 1,646
Reputation: 6,596

Next

Return to How do I....?

Who is online

Users browsing this forum: No registered users and 9 guests