Crawlit - Dofus & Dofus-Touch encyclopedia parser - Build your own Dofus API

In order to respond to a need expressed by the community, I create an encyclopedia parser for Dofus available for :

• Windows
• OSX
• Linux

Download : All versions are available here

:warning: Don't use the tool for more than 700 items per hour (for Dofus-Touch).

Crawlit was built to be very light, it can run on almost every configurations.

In fact Crawlit is a tool :

• Developed by the dofus community, for the community
• 100% performance : Developed to be very light
• Simple to use | Plug and use
• Resume parsing feature after errors (connection off / 429 error / ...)
• App packaged into executable, no need to install something
• English & French encyclopedia supported (french item & english item)

App launched | App at the end :-------------------------:|:-------------------------: |

JSON files also provided in the repository

In case you wan't directly the dofus JSON files, I oftenly upload up-to-date encyclopedia JSON in the data/ folder.

JSON format

An item result example

{  
  "_id":"item ID",
  "name":"item name",
  "description":"item description.",
  "lvl":"item lvl",
  "type":"item type",
  "imgUrl":"image url of the item",
  "url":"Item's link",
  "stats":[ "many statistics line", "stat 2", ["..."], "stat n" ],
  "condition":[ "many conditions line", "condition 2", ["..."], "condition n" ],
  "set":{  
     "equipments":[itemId, itemId ...],
     "weapons":[itemId, itemId ...],
  }
}

Getting Started

[With Npm] Install globally crawlit :

npm install -g @sosnoob/crawlit
crawlit

[With github] Clone & access this repository locally :

git clone https://github.com/raczak/crawlit-dofus-encyclopedia-parser.git
cd crawlit-dofus-encyclopedia-parser

Install dependencies with npm :

npm install

:warning: There is an issue with npm start on some CLI, in this case prefere use node lib/app.js.

[Optional] To build packaged app

If you want to generate packaged app (executable) with pkg dependency, you MUST install zeit/pkg in npm global context.
Please follow Pkg documentation for troubleshooting.

npm install -g pkg

• in a terminal window -> npm run pkg-all

Supported items

| Items | state of progress | | ------------- |:-------------| | Equipment | 100 % | | Weapon | 100 % | | Set | 100 % | | Pet | 100 % | | Mount | 100 % | | Resource | 100 % | | Consumable | 100 % | | Recipe | 25 % | | Classe | 0 % | | Professions | 0 % | | Bestiary | 0 % |

We also have done an API !

Visite the official non-official Dofus API : DOFAPI

Technologies choice

Crawlit is build with full javascript : NodeJs/Npm, javascript.

Bug & Crash || Proposal ?

For proposal don't hesitate to create a pool request or for a bug/crash an issue. You can also make us aware on bugs channel in discord.

Discord

Click here to join the sosnoob community on discord !

Documentation(dev)

Access the wiki to understand the project and contribute to it. There is also a french version of the documentation :smiley: !

Join the dev Team ?

Contact us on discord (channel general or private message to theukid)

Contributors

| Zakaria RACHEDI | Le Corre Julien | Yannick Milanetto | :---: | :---: | :---: |