node-xpt2json

The SAS® Transport File Format (XPT) is an openly documented specification maintained by SAS®, The SAS® Version 5 (V5) transport file format is an open standard developed by SAS to support data transfers between systems, especially those running different operating systems. The V5 transport specification is nonproprietary.

SAS Version 5 (V5)传输文件格式是由 SAS 开发的开放标准，用于支持系统之间的数据传输，特别是运行不同操作系统的系统之间的数据传输。

SAS® has recently developed a new transport file format specification, along with three macros, that extends the older V5 transport file format and provides new capabilities. Hence The V5 and V8 transport file is widely used in the bio-pharmaceutical industries.

V5 传输规范是非专有的。SAS 最近开发了一个新的传输文件格式规范，以及三个宏，扩展了旧的 V5 传输文件格式并提供了新的功能。因此，V5 和 V8 传输文件被广泛应用于生物制药行业。

Ever had the urge to parse XPT without SAS? And wanted to access the data in some sane, easy way? XPT file occupy too much memory in you computer and you want to compress it without any loss. Do you want to be able to display the data only through your browser? Don't want to rent a SAS licence, for whatever reason? Then xpt2json is what you're looking for!

曾经有过在没有 SAS 的情况下解析 XPT 的冲动吗？并希望以一些合理、简单的方式来访问数据？XPT 文件在你的计算机中占据了太多的内存，你想在没有任何损失的情况下进行压缩。你想只通过你的浏览器就能显示数据？不管什么原因，不想租一个 SAS 许可证？那么 xpt2json 就是你要找的解决方案!

Dataset-JSON

Year 2023，CDISC and PHUSE are delighted to announce a new pilot project aimed at supporting the adoption of Dataset-JSON as an alternative transport format for regulatory submissions. This pilot builds upon the considerable amount of work done over the years to replace XPT as the default file format for clinical and device data submissions to regulatory authorities.

2023 年，CDISC 和 PHUSE 很高兴地宣布一项新的试点项目，该项目旨在支持采用 Dataset-JSON 作为监管提交文件的替代传输格式。多年来，为取代 XPT 作为向监管机构提交临床和器械数据的默认文件格式，双方做了大量工作，而此次试点项目正是在此基础上开展的。

The pilot will be split into short-term goals of the acceptance of Dataset-JSON as a transport format option (in addition to existing XPT format), as well as the development of the future strategy relating to the adoption of advanced Dataset-JSON. An initial report is planned for Q1 2024.

试点工作将分为两个短期目标，一是接受 Dataset-JSON 作为传输格式选项（除现有的 XPT 格式外），二是制定与采用高级 Dataset-JSON 有关的未来战略。计划于 2024 年第一季度提交初步报告。'

Now, xpt2json is fully capable of converting xpt v5 and v8 data directly into Dataset-json format, so you don't need to worry about the implementation of the new standard.

现在,xpt2json 已经完全能够将 xpt v5 以及 v8 的数据直接转换为符合 Dataset-json 格式的数据，您不需要为新标准的实施有任何顾虑。

Description

Simple XPT to JavaScript object converter. I'm trying to make it support bidirectional conversion. So far, English writing has been implemented in the Node.js environment, and Chinese and browser-side will be coming soon.

简单的 XPT 到 JavaScript 对象转换器。我努力使它支持双向转换。到目前为止, Node.js 环境下已经实现英文写入，中文及浏览器端不久之后也会实现.

Installation

Simplest way to install xpt2json is to use npm, just npm install xpt2json which will download xpt2json and all dependencies.

安装 xpt2json 最简单的方法是使用 npm，只要 npm install xpt2json 就可以下载 xpt2json 和所有依赖项。

xpt2json is also available via Bower, just bower install xml2js which will download xml2js and all dependencies.

xpt2json 也可以通过 Bower 获得，只需用 bower 安装 xml2js 就可以下载 xml2js 和所有的依赖项。

Basic Read in Browser

<script src="./node_modules/xpt2json/dist/xpt2json.js"></script>
<script>
    function getFile() {
        let uploadFile = document.getElementById("upload_file_id");
        let reader = new FileReader();
        reader.readAsBinaryString(uploadFile.files[0]);
        reader.onload = (res) => {
            let jsonFile = xpt2json.readXpt(res.target.result, {
                input: "utf8",
                output: "meta|dataframe|cdisc|phuse",
            });
            console.log(jsonFile);
        };
    }
</script>

Best Practice

XPT Viewer

Basic Read in Node

const { readXpt } = require("./xpt2json");

let strPathName = "./demo/dm_zh.xpt";
let jsonData = readXpt(strPathName);
// let jsonData = readXpt(strPathName,{encoding:'utf8'});
console.log(jsonData);

Basic Write in Node

const { writeXpt } = require("./xpt2json");

let strPathName = "./demo/dm.xpt";
let jsonData = readXpt(strPathName);
// let jsonData = readXpt(strPathName,{encoding:'utf8'});
console.log(jsonData);
/*Processing*/
writeXpt("./demo/dm_test.xpt", jsonData);

Data structrue of XPT objects

We offer 3 output structures, "meta","dataframe","cdisc|phuse".

我们提供三种输出结构, "meta","dataframe" 和"cdisc|phuse";

"meta":similar to the original xpt format.

"meta": 类似于 xpt 的原生结构。

    XPT {
        VERSION:{
            type:string,
            value:"V5"|"V8"
        },
        ENCODING:{
                type:string,
                value:"ANSI"|"zeuc"|...
        },// Only for CPORT Result.
        LIBRARY:{
            type:object // data sorce information.
        },
        MEMBER:{
            type:object // dataset information, e.g. name, label
        },
        OBS:{
            type:array // Observations infrmation by variale.
            value:[
                    {
                        "NTYPE": 2,                  /* VARIABLE TYPE: 1=NUMERIC, 2=CHAR */
                        "NHFUN": 0,                  /* HASH OF NNAME (always 0) */
                        "NLNG": 7,                   /* LENGTH OF VARIABLE IN OBSERVATION e.g. length("CDISC01")=7*/
                        "NVAR0": 1,                  /* VARIABLE ORDER */
                        "NNAME": "STUDYID",          /* NAME OF VARIABLE */
                        "NLABEL": "Study Identifier",/* LABEL OF VARIABLE */
                        "NFORM": null,               /* NAME OF FORMAT */
                        "NFL": 0,                    /* FORMAT FIELD LENGTH OR 0 */
                        "NFD": 0,                    /* FORMAT NUMBER OF DECIMALS */
                        "NFJ": 0,                    /* 0=LEFT JUSTIFICATION, 1=RIGHT JUST */
                        "NFILL": null,               /* (UNUSED, FOR ALIGNMENT AND FUTURE) */
                        "NIFORM": null,              /* NAME OF INPUT FORMAT */
                        "NIFL": 0,                   /* INFORMAT LENGTH ATTRIBUTE */
                        "NIFD": 0,                   /* INFORMAT NUMBER OF DECIMALS */
                        "NPOS": 0,                   /* POSITION OF VALUE IN OBSERVATION */
                        "REST": null,                /* remaining fields are irrelevant */
                        "RECORDS": [                 /* Observation */
                                    "CDISC01",
                                    "CDISC01",
                                    "CDISC01",
                                    "CDISC01",
                                    "CDISC01"
                                    ]
                    },
                    {
                        "NTYPE": 2,
                        "NHFUN": 0,
                        "NLNG": 4,
                        "NVAR0": 2,
                        "NNAME": "SEX",
                        "NLABEL": "性别",
                        "NFORM": null,
                        "NFL": 0,
                        "NFD": 0,
                        "NFJ": 0,
                        "NFILL": null,
                        "NIFORM": null,
                        "NIFL": 0,
                        "NIFD": 0,
                        "NPOS": 786432,
                        "REST": null,
                        "RECORDS": ["男", "女", "女", "女", "男"]
                    },
                    {
                        "NTYPE": 1,
                        "NHFUN": 0,
                        "NLNG": 8,
                        "NVAR0": 3,
                        "NNAME": "AGE",
                        "NLABEL": "年龄",
                        "NFORM": null,
                        "NFL": 0,
                        "NFD": 0,
                        "NFJ": 0,
                        "NFILL": null,
                        "NIFORM": null,
                        "NIFL": 0,
                        "NIFD": 0,
                        "NPOS": 1048576,
                        "REST": null,
                        "RECORDS": [14, 13, 13, 14, 14]
                    },
                    ...
            ]
        }
}

"dataframe": A dataframe structure that is easier for python and R to use directly.

"dataframe": 更方便 python 和 R 直接使用的数据框结构。

{
    "name": "CLASS_V8",
    "label": "",
    "data": [
        {
            "var1111111": "阿尔弗雷德",
            "Name": "阿尔弗雷德",
            "Sex": "男",
            "Age": 14,
            "Height": 69,
            "Weight": 112.5
        },
        {
            "var1111111": "爱丽丝",
            "Name": "爱丽丝",
            "Sex": "女",
            "Age": 13,
            "Height": 56.5,
            "Weight": 84
        },
        {
            "var1111111": "芭芭拉",
            "Name": "芭芭拉",
            "Sex": "女",
            "Age": 13,
            "Height": 65.3,
            "Weight": 98
        },
        
    ],
    "property": [
        {
            "id": 1,
            "variable": "var1111111",
            "type": "character",
            "length": 10,
            "label": "零一二三四五六七八九零一二三四五六七八九零一二三四五六七八九零一二三四五六七八九"
        },
        {
            "id": 2,
            "variable": "name",
            "type": "character",
            "length": 12,
            "label": "姓名"
        },
        {
            "id": 3,
            "variable": "sex",
            "type": "character",
            "length": 4,
            "label": "性别"
        },
        {
            "id": 4,
            "variable": "age",
            "type": "numeric",
            "length": 8,
            "label": "年龄"
        },
        {
            "id": 5,
            "variable": "height",
            "type": "numeric",
            "length": 8,
            "label": "身高（英寸）"
        },
        {
            "id": 6,
            "variable": "weight",
            "type": "numeric",
            "length": 8,
            "label": "体重（磅）"
        }
    ]
}

"cdisc"|"phuse": Fully compliant with dataset-json format。

"cdisc"|"phuse": 完全遵循 Dataset-json 格式。

{
    "creationDateTime": "2023-08-15T07:10:13",
    "datasetJSONVersion": "1.0.0",
    "fileOID": "www.ctrial.cc",
    "asOfDateTime": "2023-07-27T12:25:15",
    "originator": "",
    "sourceSystem": "",
    "sourceSystemVersion": "",
    "clinicalData": {
        "studyOID": "",
        "metaDataVersionOID": "",
        "metaDataRef": "",
        "itemGroupData": {
            "IG.CLASS_V8": {
                "records": 6,
                "name": "CLASS_V8",
                "label": "",
                "items": [
                    {
                        "OID": "IT.VAR1111111",
                        "name": "VAR1111111",
                        "label": "零一二三四五六七八九零一二三四五六七八九零一二三四五六七八九零一二三四五六七八九",
                        "type": "character",
                        "length": 10,
                        "keySequence": 1
                    },
                    {
                        "OID": "IT.NAME",
                        "name": "NAME",
                        "label": "姓名",
                        "type": "character",
                        "length": 12,
                        "keySequence": 2
                    },
                    {
                        "OID": "IT.SEX",
                        "name": "SEX",
                        "label": "性别",
                        "type": "character",
                        "length": 4,
                        "keySequence": 3
                    },
                    {
                        "OID": "IT.AGE",
                        "name": "AGE",
                        "label": "年龄",
                        "type": "numeric",
                        "length": 8,
                        "keySequence": 4
                    },
                    {
                        "OID": "IT.HEIGHT",
                        "name": "HEIGHT",
                        "label": "身高（英寸）",
                        "type": "numeric",
                        "length": 8,
                        "keySequence": 5
                    },
                    {
                        "OID": "IT.WEIGHT",
                        "name": "WEIGHT",
                        "label": "体重（磅）",
                        "type": "numeric",
                        "length": 8,
                        "keySequence": 6
                    }
                ],
                "itemData": [
                    ["阿尔弗雷德", "阿尔弗雷德", "男", 14, 69, 112.5],
                    ["爱丽丝", "爱丽丝", "女", 13, 56.5, 84],
                    ["芭芭拉", "芭芭拉", "女", 13, 65.3, 98],
                    ["凯露", "凯露", "女", 14, 62.8, 102.5],
                    ["亨利", "亨利", "男", 14, 63.5, 102.5],
                    ["詹姆斯", "詹姆斯", "男", 12, 57.3, 83],
                    ["简", "简", "女", 12, 59.8, 84.5],
                    ["雅妮特", "雅妮特", "女", 15, 62.5, 112.5],
                    ["杰弗瑞", "杰弗瑞", "男", 13, 62.5, 84],
                    ["约翰", "约翰", "男", 12, 59, 99.5],
                    ["乔伊斯", "乔伊斯", "女", 11, 51.3, 50.5],
                    ["茱迪", "茱迪", "女", 14, 64.3, 90],
                    ["罗伊斯", "罗伊斯", "女", 12, 56.3, 77],
                    ["玛丽", "玛丽", "女", 15, 66.5, 112],
                    ["菲利普", "菲利普", "男", 16, 72, 150],
                    ["罗伯特", "罗伯特", "男", 12, 64.8, 128],
                    ["罗纳德", "罗纳德", "男", 15, 67, 133],
                    ["托马斯", "托马斯", "男", 11, 57.5, 85],
                    ["威廉", "威廉", "男", 15, 66.5, 112]
                ]
            }
        }
    }
}

Huge XPT file

Each dataset should be provided in a single transport file. The maximum size of an individual dataset that FDA can process depends on many factors. Datasets greater than 5 gigabytes (GB) in size should be split into smaller datasets no larger than 5 GB. STUDY DATA TECHNICAL CONFORMANCE GUIDE

每个数据集应在一个单一的运输文件中提供。FDA 可以处理的单个数据集的最大尺寸取决于许多因素。大于 5GB 的数据集应被分割成不大于 5GB 的小数据集。研究数据技术一致性指南。

const { readHugeXpt } = require("./`xpt2json`");
readHugeXpt("./demo/dm.xpt", {}, (data) => {
    console.log(data);
});

Kindly Reminder

There are two subtypes of SAS XPORT files in use, Version 5 and Version 8.For FDA,EMA and PMDA, you should select the V5, Both of V5 and V8 are accepted by NMPA.

目前使用的 SAS XPORT 文件有两种子类型，即第 5 版和第 8 版.若递交于 FDA，EMA 和 PMDA, 你应该选择 V5 格式，' V5 '和' V8 '都可以被 NMPA 接受。

but SAS has two forms of transport file. described here is the publicly documented transport format produced by the XPORT engine and the COPY procedure (PROC COPY) or the LOC2XPT macro. xpt2json will return a object.

但是 SAS 有两种形式的传输文件。这里描述的是由 XPORT 引擎和 COPY 过程(PROC COPY)或 LOC2XPT 宏生成的公开记录的传输格式。xpt2json 将返回一个对象。

    XPT {
        VERSION:{
            type:string,
            value:"V5"|"V8"
        },
        LIBRARY:{
            type:object // data sorce information
        },
        MEMBER:{
            type:object // dataset information, e.g. name, label
        },
    }

A second form of transport file is produced by the CPORT procedure, which is intended only for transfer between SAS versions running compatible versions of the CPORT and CIMPORT procedures. SAS Transport files processed by the CPORT SAS PROC cannot be processed or archived by the FDA. xpt2json will return a array.

第二种形式的传输文件由 CPORT 过程生成，它只用于运行 CPORT 和 CIMPORT 过程兼容版本的 SAS 版本之间的传输。由 SAS PROC CPORT 处理的传输文件不能被 FDA 处理或归档。xpt2json 将返回一个数组。

    [
        {
            ENCODING:{
                type:string,
                value:"ANSI"|"zeuc"|...
            },
            LIBRARY:{
                type:object // data sorce information
            },
            MEMBER:{
                type:object // dataset information, e.g. name, label
            },
        }
    ]
}

Getting support

Please, if you have a problem with the library, please make sure your problem really is with xpt2json. It is? Okay, then I'll look at it. Send me a mail and we can talk.

如果你的库有问题，请确定你的问题真的是出在xpt2json上。如果是的, 我一定会解决。请给我发个邮件，顺便我们可以聊聊。

You are free to use this program to develop your system. However, please do not quote the object structure or derive this data structure for derivative structure development without the permission of the author, which has been submitted to the State Intellectual Property Office of the People's Republic of China.

您可以自由应用此程序开发您的系统。但请勿在未经作者允许时引用对象结构或衍生此数据结构做衍生结构开发，此结构已递交中华人民共和国国家知识产权局。

Author: Jianfeng YE (冶剑锋)
Phone: +86 18710106985
Mail: [email protected]
linkin: www.linkedin.com/in/jianfeng-ye-63015b102

Keywords

SAS XPT JSON FDA EMA PMDA NMPA