top button
Flag Notify
    Connect to us
      Site Registration

Site Registration

How to build webcrawler in ruby?

+1 vote
402 views

I got succeed in "gem install anemone" then what's my next step to build a Web crawler.

posted Oct 11, 2013 by Seema Siddique

Share this question
Facebook Share Button Twitter Share Button LinkedIn Share Button
Now, go read the gems documentation and try out the code.

1 Answer

0 votes

If you want to write a crawler yourself you could start with hpricot.com.

answer Oct 12, 2013 by Deepankar Dubey
Similar Questions
+1 vote

In my local machine I had input some data into database using SQLite3 . When I deployed it to heroku why I can't get data that i input?

I'm doing like this

-in Gemfile
group :development do
 gem 'sqlite3'
end
group :production do
 gem 'pg'
 gem 'rails_12factor'
end
-bundle install
-heroku login
-git init 
-git add .
-git commit -m "test"
-git create
-git push heroku master
-heroku run rake db:migrate

How can I get some data when I deployed it to heroku? Please help me.

+2 votes

I am trying to search the phrase of numbers in a html page in the
sentence below:

(253 items)

I used this piece of code, but it does not work,

limit= page.search("div[class=Results]").search("div").gsub("items","")

 begin
 Integer(limit)
 rescue
 return 0
 end

Would you give me any suggestion on this?

+1 vote

Follow the code :-

class Sample
 def bar
 p method(__method__).receiver # only `self` would do the trick.
 end
end

n1=Sample.new # => #
n2=Sample.new # => #

n1.bar # => #
n2.bar # => #

But this is my try. I am looking for any method is available in Ruby,into which say if I pas n1.object_id or #,I will get back in return the local variable name say here it is - n1 ?

+2 votes
import json,httplib,urllib
connection = httplib.HTTPSConnection('api.parse.com', 443)
params =
urllib.urlencode({"username":"cooldude6","password":"p_n7!-e8"})
connection.connect()
connection.request('GET', '/1/login?%s' % params, '', {
 "X-Parse-Application-Id": "${APPLICATION_ID}",
 "X-Parse-REST-API-Key": "${REST_API_KEY}",
 "X-Parse-Revocable-Session": "1"
 })
result = json.loads(connection.getresponse().read())
print result

I need ruby equivalent code for this python script

...