spidergraph
v0.1.0
Published
GraphQL based web scraper / spider
Downloads
5
Maintainers
Readme
SpiderGraph
GraphQL based web scraper / crawler / spider
You can also find me on twitter at @skorfmann.
Example
CLI
Scrape
spidergraph scrape https://www.immobilienscout24.de/expose/54174187,https://www.immonet.de/angebot/33189259 | jq
[
{
"info": {
"query": "immobilienscout24",
"url": "https://www.immobilienscout24.de/expose/54174187"
},
"data": {
"realEstate": [
{
"__typename": "House",
"title": "Einfamilienhaus, \"Exclusive Stadtvilla\" HH Poppenbüttel",
"price": "2.700 €"
}
]
}
},
{
"info": {
"query": "immonet",
"url": "https://www.immonet.de/angebot/33189259"
},
"data": {
"realEstate": [
{
"__typename": "House",
"title": "2 Zimmer Maisonettewohnung - Neubau im Universitätsviertel",
"price": "1390.0 €"
}
]
}
}
]
Crawl
spidergraph crawl https://www.immobilienscout24.de/Suche/S-T/Wohnung-Miete/Hamburg/Hamburg
[{"info":{"query":"immobilienscout24","url":"https://www.immobilienscout24.de/expose/89156209"},"data":{"realEstate":[{"__typename":"House","title":"Wellingsbüttel - 2 Zimmer","price":"1.093,63 €"}]}}]
[{"info":{"query":"immobilienscout24","url":"https://www.immobilienscout24.de/expose/102498094"},"data":{"realEstate":[{"__typename":"House","title":"Ruhiges Wohnen mit Kamin in zentraler Lage","price":"1.863 €"}]}}]
[{"info":{"query":"immobilienscout24","url":"https://www.immobilienscout24.de/expose/99130149"},"data":{"realEstate":[{"__typename":"House","title":"Hochwertig sanierte Wohnung im Zweifamilienhaus mit Garten","price":"3.800 €"}]}}]
...
License
MIT License. See the LICENSE file.