npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@louisle2/cortex

v0.0.14

Published

<p align="center"> <img alt="cortex-cpplogo" src="https://raw.githubusercontent.com/janhq/cortex/dev/assets/cortex-banner.png"> </p>

Downloads

6

Readme

Cortex - CLI

⚠️ Cortex is currently in Development: Expect breaking changes and bugs!

About

Cortex is an openAI-compatible local AI server that developers can use to build LLM apps. It is packaged with a Docker-inspired command-line interface and a Typescript client library. It can be used as a standalone server, or imported as a library.

Cortex currently supports two inference engines:

  • Llama.cpp
  • TensorRT-LLM

Read more about Cortex at https://jan.ai/cortex

Quicklinks

Cortex

Prerequisites

Dependencies

Before installation, ensure that you have installed the following:

  • Node.js: Required for running the installation.
  • NPM: Needed to manage packages.
  • CPU Instruction Sets: Available for download from the Cortex GitHub Releases page.

💡 The CPU instruction sets are not required for the initial installation of Cortex. This dependency will be automatically installed during the Cortex initialization if they are not already on your system.

Hardware

Ensure that your system meets the following requirements to run Cortex:

  • OS:

    • MacOSX 13.6 or higher.
    • Windows 10 or higher.
    • Ubuntu 12.04 and later.
  • RAM (CPU Mode):

    • 8GB for running up to 3B models.
    • 16GB for running up to 7B models.
    • 32GB for running up to 13B models.
  • VRAM (GPU Mode):

    • 6GB can load the 3B model (int4) with ngl at 120 ~ full speed on CPU/ GPU.
    • 8GB can load the 7B model (int4) with ngl at 120 ~ full speed on CPU/ GPU.
    • 12GB can load the 13B model (int4) with ngl at 120 ~ full speed on CPU/ GPU.
  • Disk: At least 10GB for app and model download.

Quickstart

To install Cortex CLI, follow the steps below:

  1. Install the Cortex NPM package globally:
npm i -g @janhq/cortex
  1. Download a GGUF model from Hugging Face:
# Pull a model most compatible with your hardware
cortex pull llama3

# Pull a specific variant with `repo_name:branch`
cortex pull llama3:7b

# Pull a model with the HuggingFace `model_id`
cortex pull microsoft/Phi-3-mini-4k-instruct-gguf
  1. Load the model:
cortex models start llama3:7b
  1. Start chatting with the model:
cortex chat tell me a joke

Run as an API server

To run Cortex as an API server:

cortex serve

Build from Source

To install Cortex from the source, follow the steps below:

  1. Clone the Cortex repository here.
  2. Navigate to the cortex-js folder.
  3. Open the terminal and run the following command to build the Cortex project:
npx nest build
  1. Make the command.js executable:
chmod +x '[path-to]/cortex/cortex-js/dist/src/command.js'
  1. Link the package globally:
npm link

Cortex CLI Command

The following CLI commands are currently available:

⚠️ Cortex is currently in Development: More commands will be added soon!


  serve               Providing API endpoint for Cortex backend
  chat                Send a chat request to a model
  init|setup          Init settings and download cortex's dependencies
  ps                  Show running models and their status
  kill                Kill running cortex processes
  pull|download       Download a model. Working with HuggingFace model id.
  run [options]       EXPERIMENTAL: Shortcut to start a model and chat
  models              Subcommands for managing models
  models list         List all available models.
  models pull         Download a specified model.
  models remove       Delete a specified model.
  models get          Retrieve the configuration of a specified model.
  models start        Start a specified model.
  models stop         Stop a specified model.
  models update       Update the configuration of a specified model.
  engines             Execute a specified command related to engines.
  engines list        List all available engines.

Uninstall Cortex

Run the following command to uninstall Cortex globally on your machine:

# Uninstall globally using NPM
npm uninstall -g @janhq/cortex

Contact Support

  • For support, please file a GitHub ticket.
  • For questions, join our Discord here.
  • For long-form inquiries, please email [email protected].