@aghajari/jssoup

v1.0.2

Published

3 years ago

JSSoup: the JavaScript HTML DOM parser for node.js

Downloads

0High
0Medium
0Low

aghajari

jssoup htmlparser dom html

JSSoup

JSSoup is a fast and reliable HTML DOM parser library for JavaScript, node.js based on PHP: simplehtmldom and Java: Jsoup

Works with well-formed and broken HTML documents.
Loads webpages and document strings.
Supports CSS selectors.

Usage

Installation

npm i @aghajari/jssoup

const jssoup = require('@aghajari/jssoup');

Let's fetch music lyrics from Google :

const doc = await jssoup.loadFromURL("https://www.google.com/search?q=Hello+lyrics", options())

console.log('track: ' + doc.getElementByAttr("data-attrid", `"title"`).plainText())
console.log('artist' + doc.getElementByAttr("data-attrid", `"subtitle"`).plainText())
console.log('lyrics' + doc.getElementByAttr("data-lyricid").plainText())

Output:

track: Hello
artist: Adele
lyrics: Hello, it's me 
I was wondering if after all these years you'd like to meet 
To go over everything 
They say that time's supposed to heal ya...

For finding the correct css selector for an specific element, you can use HTMLNode.cssSelector(),

cssSelector(): Get a CSS selector that will uniquely select this element

Just try one example as a test and get the cssSelector for all.

const doc = await jssoup.loadFromURL("https://www.google.com/search?q=someone+like+you+lyrics", options())
console.log(doc.matchesPlainText('Someone Like You')[0].cssSelector())

You will get h2[data-attrid=title] as the cssSelector, Now use this selector for all lyrics pages from Google.

const doc = await jssoup.loadFromURL("https://www.google.com/search?q=million+years+ago+lyrics", options())
console.log(doc.findFirst('h2[data-attrid=title]').plainText()) // Output: Million Years Ago

Let's try parsing document string :

const html = `<html><head><title><b>JSSoup</b> - node.js</title></head></html>`
const title = jssoup.load(html).getElementByTagName('title') // or .findFirst('title');
console.log(title.plainText()) // JSSoup - node.js
console.log(title.innerText()) // <b>JSSoup</b> - node.js
console.log(title.outerText()) // <title><b>JSSoup</b> - node.js</title>

By the way, you can also use const title = jssoup.load(html).titleEl() for getting title.

Let's use id for finding the element.

const html = `<html><head><test id='element_id'>This is a test</test></head></html>`
const element = jssoup.load(html).getElementById('element_id') // or .findFirst('#element_id')
console.log(element) // This is a test

Let's use className for finding the element.

const html = `<html><head><test class='test header'>This is a test</test></head></html>`
const element = jssoup.load(html).getElementByClassName('test') // or .findFirst('.header')
console.log(element) // This is a test

Let's use attribute for finding the element.

const html = `<html><head><test data-type='test'>This is a test</test></head></html>`;
const element = jssoup.load(html).getElementByAttr('data-type'); // or .findFirst('[data-type]');
console.log(element); // This is a test

Other ways to get the same element :

const html = `<html><head><test class='tcls header' id='#element_id' data-type='test_attr'>This is a test</test></head></html>`
const doc = jssoup.load(html)
element = doc.getElementByTagName('test')
element = doc.findFirst('test')
element = doc.getElementByClass('tcls header')
element = doc.findFirst('test.tcls.header')
element = doc.getElementById('element_id')
element = doc.findFirst('test#element_id')
element = doc.getElementByAttr('data-type', 'test_attr')
element = doc.findFirst('test[data-type=test_attr]')
element = doc.findFirst('html > head > test:nth-child(0)')

Get comments :

const html = `<html>
<body>
    <h1>This is a test</h1>
    <!-- this is a comment -->
</body>
</html>`;

const doc = jssoup.load(html)
console.log(doc.comments()[0]) // this is a comment

Get metta tags :

const doc = await jssoup.loadFromURL("https://github.com/Aghajari", options())

console.log(doc.metaTags()) // Array of meta tags
console.log(doc.metaEl('description')) // meta element for description
console.log(doc.metaEl('image')) // meta content for image
/*
* metaEl() will search following tags:
* <meta name="NAME" content="bla bla"> (Standard)
* <meta property="og:NAME" content="bla bla">
* <meta itemprop="NAME" content="bla bla">
* <meta name="…NAME…" content="bla bla">
* <meta property="…NAME…" content="bla bla">
* <meta itemprop="…NAME…" content="bla bla">
*/

LinkPreview inspired by linkpreview :

const doc = await jssoup.loadFromURL("https://github.com/Aghajari", options())

console.log('title', doc.title()) // Aghajari - Overview
console.log('description', doc.description()) // Aghajari has ? repositories available. Follow their code on GitHub.
console.log('image', doc.image()) // https://avatars.githubusercontent.com/u/30867537?v=4?s=400

Getting multiple tags :

const html = `<html>
<body>
<tag1> This is test1 </tag1>
<tag2> This is test2 </tag2>
<tag3> This is test3 </tag3>
</body>
</html>`;

const doc = jssoup.load(html)
console.log(doc.find(['tag1', 'tag2', 'tag3'])) // Array of elements

Limit output indexes :

const html = `<html>
<body>
<tag> This is test0 </tag>
<tag> This is test1 </tag>
<tag> This is test2 </tag>
<tag> This is test3 </tag>
<tag> This is test4 </tag>
<tag> This is test5 </tag>
</body>
</html>`;

const doc = jssoup.load(html)
console.log(doc.find('tag', [2, -2])) // Array of elements (test2, test4)

Get attribute from element:

const html = `<html><head><test data-id='id1234'>This is a test</test></head></html>`;
const element = jssoup.load(html).getElementByAttr('data-id'); // or .findFirst('test[data-id]');
console.log(element.getAttribute('data-id')); // id1234

Attribute Expression

= : equal [attr=value]
!= : unequal [attr!=value]
*= : regex [attr*=value]
^= : regex /^pattern/ [attr^=value]
$= : regex /pattern$/ [attr$=value]
|= : startsWith [attr|=value]
&= : endsWith [attr&=value]
%= : contains [attr%=value]
~= : contains in list of words [attr~=value]<tag attr='blue red green'> : [attr~=red]

Author

Amir Hossein Aghajari

License

Copyright 2021 Amir Hossein Aghajari
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Published

Vulnerabilities

Links

Maintainers

Keywords