subao-sdk
v2.0.1
Published
A powerful web report package.
Downloads
1
Readme
English | 简体中文
subao-sdk is a powerful web reporting SDK that can report visitor data (such as PV and UV), performance data (including page load performance and resource data, etc.) and Error data (including resource loading errors, js runtime errors, ajax request errors, etc.). Users can use the reported data to generate statistics to help you monitor site performance, count user behaviorr, and discover unknown errors in the project.
Environment
| IE / Edge | Firefox | Chrome | Safari | Opera | Android webview | Safari on iOS | | --- | --- | --- | --- | --- | --- | --- | | IE 9, Edge 12 | 7 | 6 | 11 | 15| Yes | 11 |
Quickly start
Installation
Use npm
npm i subao-sdk
Use yarn
yarn add subao-sdk
Usage
Import as a es6 module
import { WebReport } from 'subao-sdk';
const report = new WebReport({
appId: 'application-name', // a unique key to identify the different application
domain: 'localhost:8000/report', // the target to report
});
report.init();Import as a script tag
<html>
<head>
<script src="/dist/subao-sdk.min.js"></script>
<script>
var report = new WebReport({
appId: 'application-name', // a unique key to identify the different application
domain: 'localhost:8000/report', // the target to report
});
report.init();
</script>
</head>Local debugging
Execute npm run dev in the project root directory to start the local service, and then access the test page by accessing localhost:9888 through the browser. The page has been reported to the SDK. After accessing the page, you can see the real report data of the printed browser in the console.
Configuration
domain string
The target to report. default value: ''
filterUrl string[]
Filter url that we don't want to report. default value: []
openVisitorReport boolean
Open PV/UV report or not. default value: true
openErrorReport boolean
Open error message report or not. default value: true
openPerformanceReport boolean
Open performance report or not. default value: true
appId string
Extra data including appId. default value: ''
subaoUserId string
Extra data including appId. default value: ''
loadReportDelay number
If the browser does not support the navigator.sendBeacon method, we will report the performance information of the page after the page load is delayed by loadReportDelay. default value: 5000.
urlHelper { rule: RegExp, target: string }
Replace type with the value of url in the resource and error reports. default value: { rule: /\/([a-z\-_]+)?\d{2,20}/g, target: '/$1**' }
ajaxHelper { rule: RegExp, target: string }
Replace type with the value of url in the ajax report. default value: { rule: /(\w+)\/\d{2,}/g, target: '$1' }
Generate reported data
Visitor Data
It is used to record the access information of the page. After the server obtains the data, it can be used to count the PV, UV, etc. of the page.
For UV data generation, we will add the uvExpireTime field to the localstorage, which records the timestamp of the day 23:59:59. Each visit compares the current timestamp with the timestamp saved in uvExpireTime to calculate whether to add UV.
Data Description
type
The type of data reported. value: 'pv'
appId
unique application id。value: from the Configuration
subaoUserId
unique user id。value: from the Configuration
accessId
The id of the visit. The value of this field is the same in multiple reports of the same visit. value: random string
accessTime
Current timestamp. value: timestamp
pageUrl
Full url of the page value: window.location.href
referrer
Where the page was accessed from. value: document.referrer
markUv
Whether to increase UV。value: true/false。
Example
{
accessId: "dzasARcLlHxKrmezx86DpJOzjycOgd"
accessTime: 1566462963295
appId: "111"
markUv: false
pageUrl: "http://localhost:9888/"
referrer: ""
subaoUserId: "12345"
type: "pv"
}Performance data
Performance data is used to record performance information of the page, including the time spent by the browser in processing each stage of the web page, the resources for requesting success, and the like. After the server receives the data, it can be used to analyze the page performance.
We use the performance.timing method to get the time spent on each stage of the page. For resources that are successfully requested by the page, we use the performance.getEntriesByType('resource') method to get it. Since the obtained resource data contains a lot of resources that are not helpful for performance analysis, we only intercept some of its useful fields for reporting.
Data Description
type
The type of data reported. value: 'performance'
appId
unique application id。value: from the Configuration
subaoUserId
unique user id。value: from the Configuration
accessId
The id of the visit. The value of this field is the same in multiple reports of the same visit. value: random string
timing
The time it takes for the browser to process each stage of the page. value: performance.timing
resource
The resource successfully loaded by the page. value: performance.getEntriesByType('resource')
tp
time on page。value: exit timestamp - entrance timestamp
Example
{
accessId: '16e3c8c0-ea79-11e9-b13e-ebcdab337463',
tp: 1155650,
resource:
[ { name: 'https://cdn.bootcss.com/jquery/3.4.1/core.js',
query: '?a=1&b=2',
originUrl: 'https://cdn.bootcss.com/jquery/3.4.1/core.js?a=1&b=2',
initiatorType: 'script',
duration: 0,
decodedBodySize: 9165,
nextHopProtocol: 'h2' },
{ name: 'https://cdn.bootcss.com/jquery/3.4.0/core.js',
query: '',
originUrl: 'https://cdn.bootcss.com/jquery/3.4.0/core.js',
initiatorType: 'script',
duration: 0,
decodedBodySize: 9165,
nextHopProtocol: 'h2' } ],
timing:
{ navigationStart: 1570614140718,
unloadEventStart: 1570614140729,
unloadEventEnd: 1570614140730,
redirectStart: 0,
redirectEnd: 0,
fetchStart: 1570614140719,
domainLookupStart: 1570614140719,
domainLookupEnd: 1570614140719,
connectStart: 1570614140719,
connectEnd: 1570614140719,
secureConnectionStart: 0,
requestStart: 1570614140724,
responseStart: 1570614140724,
responseEnd: 1570614140725,
domLoading: 1570614140732,
domInteractive: 1570614140758,
domContentLoadedEventStart: 1570614140758,
domContentLoadedEventEnd: 1570614140758,
domComplete: 1570614140773,
loadEventStart: 1570614140773,
loadEventEnd: 1570614140773 },
type: 'performance',
appId: '111',
subaoUserId: '12345'
}Error Data
We will listen and report the page error via the three methods window.addEventListener('error'), window.onerror, window.addEventListener('unhandledrejection').
For error catching, we prefer to use window.onerror because it is more compatible and can get detailed error information (including the rows, columns, etc. that generated the error). But when a resource (eg img, script) fails to load, the element that loads the resource triggers a error event of the Event interface and executes the onerror() on the element Processing function. The element's error event does not bubble up to window, which means it won't be caught by window.onerror.
So we need to introduce window.addEventListener('error') to catch the loading error of the resource. After that, in order to avoid repeated reporting by window.onerror and window.addEventListener('error') catching an error at the same time, we filter the event of window.addEventListener('error'). Only event.srcElement instanceof HTMLScriptElement, HTMLLinkElement, HTMLImageElement will be reported.
Data Description
type
The type of data reported. value: 'performance'
appId
unique application id。value: from the Configuration
subaoUserId
unique user id。value: from the Configuration
accessId
The id of the visit. The value of this field is the same in multiple reports of the same visit. value: random string
errorList
Error list. value: []
Example
{ accessId: '25913900-eb0c-11e9-9a93-7f3742272096',
type: 'error',
appId: '111',
subaoUserId: '12345',
errorList:
[ { url: 'http://localhost:9888/static/a.css',
originUrl: 'http://localhost:9888/static/a.css',
query: '',
time: 1570677301430,
message: 'link is loading failed',
type: 'resource' },
{ url: 'http://localhost:9888/static/a/**.js',
originUrl: 'http://localhost:9888/static/a/321321.js',
query: '',
time: 1570677301431,
message: 'script is loading failed',
type: 'resource' },
{ url: 'http://localhost:9888/static/image.png',
originUrl: 'http://localhost:9888/static/image.png?a=123',
query: '?a=123',
time: 1570677301500,
message: 'img is loading failed',
type: 'resource' },
{ url: '/static/aaa',
method: 'post',
query: '?data=123&dd=333',
originUrl: '/static/aaa/312321?data=123&dd=333',
time: 1570677301506,
message: 'fetch request error, statusText: Not Found, status: 404',
type: 'ajax' },
{ url: '/static/aaa',
method: 'get',
query: '?data=123&dd=333',
originUrl: '/static/aaa/312321?data=123&dd=333',
time: 1570677301528,
message: 'fetch request error, statusText: Not Found, status: 404',
type: 'ajax' },
{ url: '/static/bbb',
query: '?a=1&b=2',
method: 'post',
originUrl: '/static/bbb/43?a=1&b=2',
time: 1570677301534,
message: 'ajax request error, statusText: Not Found, status: 404',
type: 'ajax' }]
}Report data
Visitor report
For the user's visitor data, our goal is data volume as accurate as possible, after all, this report is used as a data source for PV, UV statistics, and The PV, UV statistical is focused on the amount and not on the quality. Therefore, we will as far as possible for the visitor data report, to avoid missing the user short-term access page. So when the SDK is initialized (init), we collect and report the visitor data to the server. In addition, in order to avoid the impact of compatibility issues as much as possible, the visitor data report uses the native ajax method.
Performance report
For performance data, we will report it on page unload, because the user's behavior will only end when unload, and the collected data is the most comprehensive. And the performance report on the page load may miss a lot of data, because even after the page load, the user's next behavior (such as: click, scroll, etc.) may trigger new resource loading and Ajax request.
In order to allow us to report data smoothly in unload, Performance reporting uses the Navigator.sendBeacon method, which will send data asynchronously to the server on page unload without delaying the unload of the page.
Navigator.sendBeacon does not support setting the Content-Type of the request header, although it can be implemented with Blob encapsulation. However, considering the Blob compatibility issue, I gave up adding Content-Type and unified the default value of text/plain for reporting.
Considering the Navigator.sendBeacon compatibility issue, someone will select the page unload to report with sync ajax when the Navigator.sendBeacon method is not supported, but this will cause the page uninstallation to be delayed, Seriously affect the user experience. Our solution is to choose to delay the actual performance of the performance after the page load. As mentioned earlier, this reporting opportunity may miss some resource data. But compared to the catastrophic impact of sync ajax on user access, we prefer to make sacrifices in reporting. Moreover, the statistics tell us that the current user equipment that already has 90% supports the Navigator.sendBeacon method. So let's ignore the subtle data differences that may be brought about for the time being.
Error report
In general, every time error is captured, it will trigger a error report, but in order to avoid excessive short-term errors (for example: errors caused by scroll events), the number of reports is excessive, we are wrong The report is anti-shake, and the error data will be added to errorList first. If there is no new error trigger within 2000ms, it will be reported. Wrong reporting will also take the form of native ajax.
Manually report errors
By reason of the error event cannot monitor the Error that has been caught, we can call the manualReportError method in the webReport instance to manually report it when we need, provided that you have instantiated WebReport. The manualReportError method will also perform debounce processing within 2000ms.
var report = new window.WebReport({
domain: '/report',
appId: '111',
subaoUserId: '12345',
filterUrl: ['/report', 'subao-sdk']
});
function myFunction() {
try {
throw new Error('caught error')
} catch(err) {
report.manualReportError({
url: 'www.renrendai.com/sw.js',
message: 'Unexpected error'
});
}
}
Reference document
https://developer.mozilla.org/zh-CN/docs/Web/API/Navigator/sendBeacon
https://developer.mozilla.org/zh-CN/docs/Web/API/GlobalEventHandlers/onerror
http://www.aliued.com/?p=4172
http://www.alloyteam.com/2015/09/explore-performance/
https://blog.seosiwei.com/detail/30