comment-chunk-helper: A utility function that makes it easier to write code chunkers for Sense dashboards.

Rationale

Common code operations like parsing and syntax highlighting have strong library support. While writing Sense dashboards, however, we found that another code operation we call chunking is often awkward to implement. Code chunking is splitting code up into logical units such as comment and statement blocks, which are the top-level executable units of Sense's interactive dashboards (REPLs). Chunking allows dashboards to display code, comments, results and output in an order that closely matches the source code. As such, the goal is stylistic rather than formal.

As an example, we might chunk JavaScript source as follows:

// Here are some line comments.
// Successive line comments should be grouped together in a chunk.
// These chunks are often markdown-formatted and displayed as documentation.

var a = 2;

var b = 2;  var c = 3;

/*
Here's a block comment. Block comments can be markdown-formatted and displayed
as documentation too.
*/

var square = function (x) {
  return x * x;
};

if (true) {
  console.log(square(3)); 
}
else {
  console.log('Not displaying the answer.');
}

Most languages have parsers available that get you most of the way to a chunker. Parsers usually output a sequence of abstract syntax tree nodes, which correspond to the code chunks, and the line and column at which each chunk begins and ends. If you can find such a parser for your language, this package will help you create a chunker relatively quickly. If your parser doesn't make a record of comments, no problem- this module will go back through the code and pick them up.

Usage

var parserFunc = function(code, cb) {
  // You have to supply this function.
  
  // If a syntax error or other error occurs during parsing, it should call
  cb("the error message")
  
  // If the code parses, it should call
  cb(false, [
    {start: {line: 0, column: 0}, end: {line: 1, column: 3}}, // the type is "code" by default.
    {start: {line: 1, column: 0}, end: {line: 10, column: 4}, type: "blockComment"}
    ...
  ])
  
  // Note that the end line and columns are exclusive upper bounds and that
  // indexing begins at zero, so the statemnt
  "var x;"
  // would be 
  {start: {line: 0, column: 0}, end: {line: 1, column: 7}}.
}

var chunk = require('comment-chunk-helper').chunk({
  parser: parserFunc,
  lineComment: "//", 
  blockComment: {left: "/*", right: "*/"}
});

chunk(code, cb);
// The single argument passed to the callback will be an array
// of chunks, that is, an argument of the form
[
  {
    type: ("code" or "error" or "comment" or "blockComment"),
    value: "the code, comment, or error message"
  }
]
// Note that the chunker's callback should not take an error
// argument. If any error occurs while chunking, the type of
// one of the returned chunks should be set to 'error'. That
// way the error information is displayed to the user of the
// dashboard.