# Table of Contents - [Danfo.js Documentation | Danfo.js](#danfo-js-documentation-danfo-js) - [danfo. convertFunctionTotransformer | Danfo.js](#danfo-convertfunctiontotransformer-danfo-js) - [General Functions | Danfo.js](#general-functions-danfo-js) - [danfo.Dt | Danfo.js](#danfo-dt-danfo-js) - [danfo.dateRange | Danfo.js](#danfo-daterange-danfo-js) - [danfo.Utils | Danfo.js](#danfo-utils-danfo-js) - [Getting Started | Danfo.js](#getting-started-danfo-js) - [danfo.Str | Danfo.js](#danfo-str-danfo-js) - [danfo.streamJSON | Danfo.js](#danfo-streamjson-danfo-js) - [danfo.tensorflow | Danfo.js](#danfo-tensorflow-danfo-js) - [API reference | Danfo.js](#api-reference-danfo-js) - [danfo.streamCSV | Danfo.js](#danfo-streamcsv-danfo-js) - [danfo.streamCsvTransformer | Danfo.js](#danfo-streamcsvtransformer-danfo-js) - [danfo.toDateTime | Danfo.js](#danfo-todatetime-danfo-js) - [danfo.MinMaxScaler | Danfo.js](#danfo-minmaxscaler-danfo-js) --- # Danfo.js Documentation | Danfo.js D**anfo.js** is heavily inspired by the [Pandas](https://pandas.pydata.org/pandas-docs/stable/index.html) library and provides a similar interface and API. This means users familiar with the [Pandas](https://pandas.pydata.org/pandas-docs/stable/index.html) API can easily use D**anfo.js.** [](#main-features) Main Features ------------------------------------- * Danfo.js is fast and supports [Tensorflow.js](https://js.tensorflow.org) 's tensors out of the box. This means you can [convert Danfo.js](/api-reference/dataframe) DataFrames to Tensors, and vice versa. * Easy handling of missing data (represented as `NaN, undefined, or null`) in data * Size mutability: columns can be inserted/deleted from DataFrames * Automatic and explicit alignment: objects can be explicitly aligned to a set of labels, or the user can simply ignore the labels and let [`Series`](/api-reference/series) , [`DataFrame`](/api-reference/dataframe) , etc. automatically align the data for you in computations * Powerful, flexible, [groupby](/api-reference/groupby) functionality to perform split-apply-combine operations on data sets, for both aggregating and transforming data * Make it easy to convert Arrays, JSONs, List or Objects, Tensors, and differently-indexed data structures into DataFrame objects * Intelligent label-based slicing, fancy indexing, and querying of large data sets * Intuitive [merging](/api-reference/general-functions/danfo.merge) and [joining](/api-reference/general-functions/danfo.concat) data sets * Robust IO tools for loading data from [flat-files](/api-reference/input-output/danfo.read_csv) (CSV and delimited), Excel, and JSON data format. * Powerful, flexible, and intiutive API for [plotting](https://app.gitbook.com/@jsdata/s/danfojs/~/drafts/-MESZnq3_VBU0EW71MxS/api-reference/plotting) DataFrames and Series interactively. * Timeseries-specific functionality: date range generation and date and time properties. * Robust data preprocessing functions like [OneHotEncoders](/api-reference/general-functions/danfo.onehotencoder) , [LabelEncoders](/api-reference/general-functions/danfo.labelencoder) , and scalers like [StandardScaler](/api-reference/general-functions/danfo.standardscaler) and [MinMaxScaler](/api-reference/general-functions/danfo.minmaxscaler) are supported on DataFrame and Series [](#getting-started) Getting Started ----------------------------------------- New to Danfo? Check out the getting started guides. It contains a quick introduction to D\_anfo's\_ main concepts and links to additional content. [](#api-reference) **API Reference** ----------------------------------------- The reference guide contains a detailed description of the **danfo** API. The reference describes how each function works and which parameters can be used. [](#user-guides-tutorials) User Guides/Tutorials ----------------------------------------------------- [](#building-data-driven-applications-with-danfo.js-book) Building Data Driven Applications with Danfo.js - Book --------------------------------------------------------------------------------------------------------------------- [](#contributing-guide) Contributing Guide ----------------------------------------------- Want to help improve our documentation and existing functionalities? The contributing guidelines will guide you through the process. [](#release-notes) Release Notes ------------------------------------- [NextGetting Started](/getting-started) Last updated 2 years ago Was this helpful? [Getting Started](/getting-started) [API reference](/api-reference) [User Guides](/examples) [Building Data Driven Applications with Danfo.js - Book](/building-data-driven-applications-with-danfo.js-book) [Contributing Guide](/contributing-guide) [Release Notes](/release-notes) --- # danfo. convertFunctionTotransformer | Danfo.js danfo.**convertFunctionTotransformer**(func) Parameters Type Description Default **func** Function **Returns:** > return A [pipe transformer](https://nodejs.org/api/stream.html#implementing-a-transform-stream) > that applies the function to each row of object. The **convertFunctionTotransformer** takes a function and converts it to a Nodejs stream transformer function which can be used in combination with streamCsvTransformer to incrementally transform large files. [](#converting-a-function-to-a-transformer) **Converting a function to a transformer** ------------------------------------------------------------------------------------------- NodeBrowser Copy const dfd = require("danfojs-node") /* * A simple function that takes each row of a DataFrame and splits the * name field. */ const renamer = (dfRow: DataFrame) => { const dfModified = dfRow["Names"].map((name) => name.split(",")[0]) return dfModified } const transformer = dfd.convertFunctionTotransformer(renamer) console.log(transformer) Copy Document Output Copy Transform { _readableState: ReadableState { objectMode: true, highWaterMark: 16, buffer: BufferList { head: null, tail: null, length: 0 }, length: 0, pipes: [], flowing: null, ended: false, endEmitted: false, reading: false, sync: false, needReadable: false, emittedReadable: false, readableListening: false, resumeScheduled: false, errorEmitted: false, emitClose: true, autoDestroy: true, destroyed: false, errored: null, closed: false, closeEmitted: false, defaultEncoding: 'utf8', awaitDrainWriters: null, writecb: null, writechunk: null, writeencoding: null } } [Previousdanfo.tensorflow](/api-reference/general-functions/danfo.tensorflow) [Nextdanfo.streamCsvTransformer](/api-reference/general-functions/danfo.streamcsvtransformer) Last updated 3 years ago Was this helpful? A valid JavaScript function to convert to a [pipe transformer.](https://nodejs.org/api/stream.html#implementing-a-transform-stream) --- # General Functions | Danfo.js ### [](#data-transformation) Data transformation Merge DataFrame or named Series objects with a database-style join. Concatenate danfo objects along a particular axis with optional set logic along the other axes. Convert categorical variable into dummy/indicator variables. Similar to OneHotEncoding ### [](#data-normalization) Data Normalization Encode target labels with value between 0 and n\_classes-1. Encode categorical features as a one-hot numeric array. Standardize features by removing the mean and scaling to unit variance Transform features by scaling each feature to a given range ### [](#working-with-datetime) Working with DateTime Convert argument to datetime. Return a fixed frequency Datetime Index. A class that converts strings of Date Time into a usable format, by exposing various helper methods. ### [](#streaming-functions) Streaming Functions A function that loads a CSV object as a stream, returning intermediate rows as a DataFrame. A function that loads a JSON object as a stream, returning intermediate rows as a DataFrame. A function that loads a CSV object as a stream, and applies a map-reduce function to intermediate rows. A function to convert any JS function into a Stream transformer. ### [](#utility-and-configurations) Utility and Configurations A utility class with helper methods mostly used internally. Base configuration class for NDframe objects ### [](#strings) Strings A class that converts strings into a usable format, by exposing various helper methods. ### [](#internal-libs) Internal Libs Exported Tensorflow.js library. This helps to avoid duplicated Tensorflow.js library use. [PreviousAPI reference](/api-reference) [Nextdanfo.tensorflow](/api-reference/general-functions/danfo.tensorflow) Last updated 3 years ago Was this helpful? [`merge`](/api-reference/general-functions/danfo.merge) [`concat`](/api-reference/general-functions/danfo.concat) [`getDummies`](/api-reference/general-functions/danfo.get_dummies) [LabelEncoder](/api-reference/general-functions/danfo.labelencoder) [OneHotEncoder](/api-reference/general-functions/danfo.onehotencoder) [StandardScaler](/api-reference/general-functions/danfo.standardscaler) [`MinMaxScaler`](/api-reference/general-functions/danfo.minmaxscaler) [`toDateTime`](/api-reference/general-functions/danfo.to_datetime) [`dateRange`](/api-reference/general-functions/danfo.date_range) [Dt](/api-reference/general-functions/danfo.dt) [streamCSV](/api-reference/general-functions/danfo.streamcsv) [streamJSON](/api-reference/general-functions/danfo.streamjson) [streamCSVTransformer](/api-reference/general-functions/danfo.streamcsvtransformer) [convertFunctionTotransformer](/api-reference/general-functions/danfo.-convertfunctiontotransformer) [Utils](/api-reference/general-functions/danfo.utils) [Config](https://github.com/javascriptdata/danfojs-doc/blob/master/api-reference/general-functions/broken-reference/README.md) [Str](/api-reference/general-functions/danfo.str) [tensoflow](/api-reference/general-functions/danfo.tensorflow) --- # danfo.Dt | Danfo.js For example, in the following example, we convert a Series to an `Dt` instance and apply a couple of **DateTime** methods. Node Copy import { Dt, Series } from "danfojs-node" const sf = new Series(["1/1/2000", "1/2/2000", "2/3/2000", "1/4/2000", "4/5/2000"]) const dtS = new Dt(sf) dtS.dayOfWeekName().print() dtS.monthName().print() Copy // output ╔═══╤═══════════╗ ║ 0 │ Saturday ║ ╟───┼───────────╢ ║ 1 │ Sunday ║ ╟───┼───────────╢ ║ 2 │ Thursday ║ ╟───┼───────────╢ ║ 3 │ Tuesday ║ ╟───┼───────────╢ ║ 4 │ Wednesday ║ ╚═══╧═══════════╝ ╔═══╤══════════╗ ║ 0 │ January ║ ╟───┼──────────╢ ║ 1 │ January ║ ╟───┼──────────╢ ║ 2 │ February ║ ╟───┼──────────╢ ║ 3 │ January ║ ╟───┼──────────╢ ║ 4 │ April ║ ╚═══╧══════════╝ [Previousdanfo.Str](/api-reference/general-functions/danfo.str) [Nextdanfo.dateRange](/api-reference/general-functions/danfo.date_range) Last updated 2 years ago Was this helpful? --- # danfo.dateRange | Danfo.js danfo.**dateRange**(options) Parameters Type Description **options** Object Includes any of the following: **start**: Left bound for generating dates. **end**: Right bound for generating dates. **period** : Number of periods to generate. **offSet**: Date range offset **freq**: Date range frequency. One of \["M","D","s","H","m","Y"\] [](#examples) **Examples** ------------------------------- NodeBrowser Copy const dfd = require("danfojs-node") let data = new dfd.dateRange({"start":'1/1/2018', period:5, freq:'M'}) console.log(data); Copy Document Output Copy [\ '1/1/2018, 12:00:00 AM',\ '2/1/2018, 12:00:00 AM',\ '3/1/2018, 12:00:00 AM',\ '4/1/2018, 12:00:00 AM',\ '5/1/2018, 12:00:00 AM'\ ] NodeBrowser Copy const dfd = require("danfojs-node") let data = new dfd.dateRange({ "start": '1/1/2018', period: 12, freq: 'Y' }) console.log(data); Copy Output Copy [\ '1/1/2018, 12:00:00 AM',\ '1/1/2019, 12:00:00 AM',\ '1/1/2020, 12:00:00 AM',\ '1/1/2021, 12:00:00 AM',\ '1/1/2022, 12:00:00 AM',\ '1/1/2023, 12:00:00 AM',\ '1/1/2024, 12:00:00 AM',\ '1/1/2025, 12:00:00 AM',\ '1/1/2026, 12:00:00 AM',\ '1/1/2027, 12:00:00 AM',\ '1/1/2028, 12:00:00 AM',\ '1/1/2029, 12:00:00 AM'\ ] Datetime properties of Series or datetime-like columns in DataFrame can be accessed via accessors in the **dt** name space. See [Accessors](https://app.gitbook.com/@jsdata/s/danfojs/~/drafts/-MEMaWwva1cjt8CxnG-b/api-reference/series#accessors) [Previousdanfo.Dt](/api-reference/general-functions/danfo.dt) [Nextdanfo.OneHotEncoder](/api-reference/general-functions/danfo.onehotencoder) Last updated 3 years ago Was this helpful? --- # danfo.Utils | Danfo.js The Utils class holds useful utility methods, mostly used internally in the Danfojs library. For example, in the following example, we use the `inferDtype` function from the utils class. Node Copy import { Utils } from "danfojs-node" const utils = new Utils() const arr = [NaN, 2.1, 3.3, 2.09] console.log(utils.inferDtype(arr)) //output [ 'float32' ] [Previousdanfo.streamCSV](/api-reference/general-functions/danfo.streamcsv) [Nextdanfo.Str](/api-reference/general-functions/danfo.str) Last updated 3 years ago Was this helpful? --- # Getting Started | Danfo.js A stable version of Danfojs (v1), has been released, and it comes with full Typescript support, new features, and many bug fixes. See release note [here](/release-notes#latest-release-node-v1.0.0-browser-v1.0.0) . There are a couple of breaking changes, so we have prepared a short migration [guide](/examples/migrating-to-the-stable-version-of-danfo.js) for pre-v1 users. [](#installation) Installation ----------------------------------- There are three ways to install and use Danfo.js in your application For Nodejs applications, you can install the [danfojs-node](https://www.npmjs.com/package/danfojs-node) version via package managers like yarn and npm: Copy npm install danfojs-node or yarn add danfojs-node For client-side applications built with frameworks like React, Vue, Next.js, etc, you can install the [danfojs](https://www.npmjs.com/package/danfojs) version: Copy npm install danfojs or yarn add danfojs For use directly in HTML files, you can add the latest script tag from [JsDelivr](https://www.jsdelivr.com/package/npm/danfojs) : Copy To play with Danfo.js in a Notebook-like environment, see [Dnotebooks](https://dnotebook.jsdata.org/getting-started) [here](https://playnotebook.jsdata.org/demo) or the [VS-Code Nodejs notebook extension](https://marketplace.visualstudio.com/items?itemName=donjayamanne.typescript-notebook) . [](#id-10-minutes-to-danfo.js) 10 minutes to danfo.js ---------------------------------------------------------- This is a short introduction to Danfo.js, and its flow is adapted from the official [10 minutes to Pandas](https://pandas.pydata.org/pandas-docs/stable/user_guide/10min.html#min) We will show you how to use danfo.js in a browser, client-side libraries, and Node.js environments. Most functions except [plotting](https://jsdata.gitbook.io/danfojs/api-reference/plotting) which require a DOM work the same way in all environments. NodeBrowserReact Copy const dfd = require("danfojs-node") //or using ES6 import * as dfd from "danfojs-node" Copy Copy import * as dfd from "danfojs" //import specific methods/classes import { readCSV, DataFrame } from "danfojs" ### [](#creating-a-dataframe-series) Creating a DataFrame/Series You can create a [`Series`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.html#pandas.Series) by passing a list of values, letting Danfo.js create a default integer index: NodeBrowser Copy import * as dfd from "danfojs-node" s = new dfd.Series([1, 3, 5, undefined, 6, 8]) s.print() Copy Document Copy //output ╔═══╤══════════════════════╗ ║ │ 0 ║ ╟───┼──────────────────────╢ ║ 0 │ 1 ║ ╟───┼──────────────────────╢ ║ 1 │ 3 ║ ╟───┼──────────────────────╢ ║ 2 │ 5 ║ ╟───┼──────────────────────╢ ║ 3 │ undefined ║ ╟───┼──────────────────────╢ ║ 4 │ 6 ║ ╟───┼──────────────────────╢ ║ 5 │ 8 ║ ╚═══╧══════════════════════╝ Creating a [`Series`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.html#pandas.Series) from a tensor NodeBrowser Copy const dfd = require("danfojs-node") const tf = dfd.tensorflow //Tensorflow.js is exportedfrom Danfojs let tensor_arr = tf.tensor([12,34,56,2]) let s = new dfd.Series(tensor_arr) s.print() Copy Document Copy ╔═══╤════╗ ║ 0 │ 12 ║ ╟───┼────╢ ║ 1 │ 34 ║ ╟───┼────╢ ║ 2 │ 56 ║ ╟───┼────╢ ║ 3 │ 2 ║ ╚═══╧════╝ Creating a [`DataFrame`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html#pandas.DataFrame) by passing a JSON object: NodeBrowser Copy const dfd = require("danfojs-node") json_data = [{ A: 0.4612, B: 4.28283, C: -1.509, D: -1.1352 },\ { A: 0.5112, B: -0.22863, C: -3.39059, D: 1.1632 },\ { A: 0.6911, B: -0.82863, C: -1.5059, D: 2.1352 },\ { A: 0.4692, B: -1.28863, C: 4.5059, D: 4.1632 }] df = new dfd.DataFrame(json_data) df.print() Copy Document Creating a [`DataFrame`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html#pandas.DataFrame) from a 2D tensor NodeBrowser Copy const dfd = require("danfojs-node") const tf = dfd.tensorflow //Tensorflow.js is exported from Danfojs let tensor_arr = tf.tensor2d([[12, 34, 2.2, 2], [30, 30, 2.1, 7]]) let df = new dfd.DataFrame(tensor_arr, {columns: ["A", "B", "C", "D"]}) df.print() df.ctypes.print() Copy Document Copy ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C │ D ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ 12 │ 34 │ 2.20000004768... │ 2 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ 30 │ 30 │ 2.09999990463... │ 7 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ ╔═══╤══════════════════════╗ ║ │ 0 ║ ╟───┼──────────────────────╢ ║ A │ int32 ║ ╟───┼──────────────────────╢ ║ B │ int32 ║ ╟───┼──────────────────────╢ ║ C │ float32 ║ ╟───┼──────────────────────╢ ║ D │ int32 ║ ╚═══╧══════════════════════╝ Creating a [`DataFrame`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html#pandas.DataFrame) by passing a dictionary of objects with the same length NodejsBrowser Copy const dfd = require("danfojs-node") // Danfojs v1.0.0 and above dates = new dfd.dateRange({ start: '2017-01-01', end: "2020-01-01", period: 4, freq: "Y" }) console.log(dates); obj_data = {'A': dates, 'B': ["bval1", "bval2", "bval3", "bval4"], 'C': [10, 20, 30, 40], 'D': [1.2, 3.45, 60.1, 45], 'E': ["test", "train", "test", "train"] } df = new dfd.DataFrame(obj_data) df.print() Copy Document Copy //output in console ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C │ D │ E ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ 1/1/2017, 1:0... │ bval1 │ 10 │ 1.2 │ test ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ 1/1/2018, 1:0... │ bval2 │ 20 │ 3.45 │ train ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ 1/1/2019, 1:0... │ bval3 │ 30 │ 60.1 │ test ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ 1/1/2020, 1:0... │ bval4 │ 40 │ 45 │ train ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ The columns of the resulting [`DataFrame`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html#pandas.DataFrame) have different [dtypes](https://pandas.pydata.org/pandas-docs/stable/user_guide/basics.html#basics-dtypes) . Copy df.ctypes.print() Copy //output ╔═══╤═════════╗ ║ A │ string ║ ╟───┼─────────╢ ║ B │ string ║ ╟───┼─────────╢ ║ C │ int32 ║ ╟───┼─────────╢ ║ D │ float32 ║ ╟───┼─────────╢ ║ E │ string ║ ╚═══╧═════════╝ Creating a [`DataFrame`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html#pandas.DataFrame) by passing an array of arrays. Index and column labels are automatically generated for you. NodeBrowser Copy const dfd = require("danfojs-node") arr_data = [["bval1", 10, 1.2, "test"],\ ["bval2", 20, 3.45, "train"],\ ["bval3", 30, 60.1, "train"],\ ["bval4", 35, 3.2, "test"]] df = new dfd.DataFrame(arr_data) df.print() Copy Document Copy //output in console ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ 0 │ 1 │ 2 │ 3 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ bval1 │ 10 │ 1.2 │ test ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ bval2 │ 20 │ 3.45 │ train ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ bval3 │ 30 │ 60.1 │ train ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ bval4 │ 35 │ 3.2 │ test ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ ### [](#viewing-data) Viewing data Here is how to view the top and bottom rows of the frame above: Copy df.head(2).print() df.tail(2).print() Copy //output from head ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ 0 │ 1 │ 2 │ 3 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ bval1 │ 10 │ 1.2 │ test ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ bval2 │ 20 │ 3.45 │ train ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ //output from tail ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ 0 │ 1 │ 2 │ 3 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ bval3 │ 30 │ 60.1 │ train ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ bval4 │ 35 │ 3.2 │ test ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ Display the index, columns: JavaScriptBrowser Copy const dfd = require('danfojs-node') let dates = new dfd.dateRange({ start: "2017-01-01", end: "2020-01-01", period: 4, freq: "Y", }); let obj_data = { A: dates, B: ["bval1", "bval2", "bval3", "bval4"], C: [10, 20, 30, 40], D: [1.2, 3.45, 60.1, 45], E: ["test", "train", "test", "train"], }; let df = new dfd.DataFrame(obj_data); df.print(); console.log(df.index); console.log(df.columns); Copy Document Copy //output ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C │ D │ E ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ 1/1/2017, 1:00:… │ bval1 │ 10 │ 1.2 │ test ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ 1/1/2018, 1:00:… │ bval2 │ 20 │ 3.45 │ train ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ 1/1/2019, 1:00:… │ bval3 │ 30 │ 60.1 │ test ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ 1/1/2020, 1:00:… │ bval4 │ 40 │ 45 │ train ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ [ 0, 1, 2, 3 ] [ 'A', 'B', 'C', 'D', 'E' ] [`DataFrame.tensor`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_numpy.html#pandas.DataFrame.to_numpy) returns a Tensorflow tensor representation of the underlying data. Note that **Tensorflow tensors have one dtype for the entire array, while danfo DataFrames have one dtype per column**. For `df`, our [`DataFrame`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html#pandas.DataFrame) of all floating-point values, [`DataFrame.tensor`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_numpy.html#pandas.DataFrame.to_numpy) is fast and doesn’t require copying data. NodeBrowser Copy const dfd = require("danfojs-node") j son_data = [{ A: 0.4612, B: 4.28283, C: -1.509, D: -1.1352 },\ { A: 0.5112, B: -0.22863, C: -3.39059, D: 1.1632 },\ { A: 0.6911, B: -0.82863, C: -1.5059, D: 2.1352 },\ { A: 0.4692, B: -1.28863, C: 4.5059, D: 4.1632 }] let df = new dfd.DataFrame(json_data) console.log(df.tensor); //or df.tensor.print() Copy Document Copy //output Tensor { kept: false, isDisposedInternal: false, shape: [ 4, 4 ], dtype: 'float32', size: 16, strides: [ 4 ], dataId: {}, id: 0, rankType: '2' } Tensor [[0.4612, 4.2828302, -1.5089999, -1.1352 ],\ [0.5112, -0.22863 , -3.39059 , 1.1632 ],\ [0.6911, -0.82863 , -1.5059 , 2.1352 ],\ [0.4692, -1.28863 , 4.5058999 , 4.1631999]] **Note** [`DataFrame.tensor`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_numpy.html#pandas.DataFrame.to_numpy) does _not_ include the index or column labels in the output. [`describe()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.describe.html#pandas.DataFrame.describe) shows a quick statistic summary of your data: NodeBrowser Copy const dfd = require("danfojs-node") let json_data = [{ A: 0.4612, B: 4.28283, C: -1.509, D: -1.1352 },\ { A: 0.5112, B: -0.22863, C: -3.39059, D: 1.1632 },\ { A: 0.6911, B: -0.82863, C: -1.5059, D: 2.1352 },\ { A: 0.4692, B: -1.28863, C: 4.5059, D: 4.1632 }] let df = new dfd.DataFrame(json_data) df.describe().print() Copy Document Copy //output in console ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C │ D ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ count │ 4 │ 4 │ 4 │ 4 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ mean │ 0.533175 │ 0.4842349999999… │ -0.474897500000… │ 1.5816 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ std │ 0.1075428712963… │ 2.5693167249095… │ 3.4371471031498… │ 2.2005448052698… ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ min │ 0.4612 │ -1.28863 │ -3.39059 │ -1.1352 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ median │ 0.4901999999999… │ -0.528629999999… │ -1.50745 │ 1.6492 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ max │ 0.6911 │ 4.28283 │ 4.5059 │ 4.1632 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ variance │ 0.0115654691666… │ 6.6013884328999… │ 11.813980208691… │ 4.84239744 ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ Sorting by values (Defaults to ascending): NodeBrowser Copy const dfd = require("danfojs") let data = {"A": [-20, 30, 47.3, NaN], "B": [34, -4, 5, 6] , "C": [20, 2, 3, 30] } let df = new dfd.DataFrame(data) df.sortValues("C", {inplace: true}) df.print() Copy Document Copy ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ 30 │ -4 │ 2 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ 47.3 │ 5 │ 3 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ -20 │ 34 │ 20 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ NaN │ 6 │ 30 ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ ### [](#selection) Selection #### [](#getting) Getting Selecting a single column, which yields a [`Series`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.html#pandas.Series) , equivalent to `df.A`: NodeBrowser Copy const dfd = require("danfojs-node") json_data = [{ A: 0.4612, B: 4.28283, C: -1.509, D: -1.1352 },\ { A: 0.5112, B: -0.22863, C: -3.39059, D: 1.1632 },\ { A: 0.6911, B: -0.82863, C: -1.5059, D: 2.1352 },\ { A: 0.4692, B: -1.28863, C: 4.5059, D: 4.1632 }] df = new dfd.DataFrame(json_data) df['A'].print() Copy Document Copy //output ╔═══╤══════════════════════╗ ║ │ A ║ ╟───┼──────────────────────╢ ║ 0 │ 0.4612 ║ ╟───┼──────────────────────╢ ║ 1 │ 0.5112 ║ ╟───┼──────────────────────╢ ║ 2 │ 0.6911 ║ ╟───┼──────────────────────╢ ║ 3 │ 0.4692 ║ ╚═══╧══════════════════════╝ #### [](#selection-by-label) Selection by label For getting a cross-section using a label: Copy const dfd = require("danfojs") let data = { "Name": ["Apples", "Mango", "Banana", "Pear"] , "Count": [21, 5, 30, 10] , "Price": [200, 300, 40, 250] } let df = new dfd.DataFrame(data, {index: ["a", "b", "c", "d"]}) df.print() let sub_df = df.loc({rows: ["a", "c"]}) sub_df.print() Copy ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Name │ Count │ Price ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ a │ Apples │ 21 │ 200 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ b │ Mango │ 5 │ 300 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ c │ Banana │ 30 │ 40 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ d │ Pear │ 10 │ 250 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╝ Shape: (2,3) ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Name │ Count │ Price ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ a │ Apples │ 21 │ 200 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ c │ Banana │ 30 │ 40 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╝ Selecting on a multi-axis by label: Copy const dfd = require("danfojs-node") let data = { "Name": ["Apples", "Mango", "Banana", "Pear"] , "Count": [21, 5, 30, 10], "Price": [200, 300, 40, 250] } let df = new dfd.DataFrame(data) df.print() let sub_df = df.loc({ rows: [0,1], columns: ["Name", "Price"] }) sub_df.print() Copy ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Name │ Count │ Price ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ Apples │ 21 │ 200 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ Mango │ 5 │ 300 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ Banana │ 30 │ 40 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ Pear │ 10 │ 250 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╝ Shape: (2,2) ╔═══╤═══════════════════╤═══════════════════╗ ║ │ Name │ Price ║ ╟───┼───────────────────┼───────────────────╢ ║ 0 │ Apples │ 200 ║ ╟───┼───────────────────┼───────────────────╢ ║ 1 │ Mango │ 300 ║ ╚═══╧═══════════════════╧═══════════════════╝ Showing label slicing: Copy const dfd = require("danfojs-node") let data = { "Name": ["Apples", "Mango", "Banana", "Pear"] , "Count": [21, 5, 30, 10], "Price": [200, 300, 40, 250] } let df = new dfd.DataFrame(data) df.print() let sub_df = df.loc({ rows: ["0:2"], columns: ["Name", "Price"] }) sub_df.print() Copy //before slicing ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Name │ Count │ Price ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ Apples │ 21 │ 200 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ Mango │ 5 │ 300 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ Banana │ 30 │ 40 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ Pear │ 10 │ 250 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╝ //after slicing ╔════════════╤═══════════════════╤═══════════════════╗ ║ │ Name │ Price ║ ╟────────────┼───────────────────┼───────────────────╢ ║ 0 │ Apples │ 200 ║ ╟────────────┼───────────────────┼───────────────────╢ ║ 1 │ Mango │ 300 ║ ╚════════════╧═══════════════════╧═══════════════════╝ #### [](#selection-by-position) Selection by position Select via the position of the passed integers: Copy const dfd = require("danfojs-node") let data = { "Name": ["Apples", "Mango", "Banana", "Pear"] , "Count": [21, 5, 30, 10] , "Price": [200, 300, 40, 250] } let df = new dfd.DataFrame(data) let sub_df = df.iloc({rows: [1,3]}) sub_df.print() Copy ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Name │ Count │ Price ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ Mango │ 5 │ 300 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ Pear │ 10 │ 250 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╝ By integer slices: Copy const dfd = require("danfojs-node") let data = { "Name": ["Apples", "Mango", "Banana", "Pear"] , "Count": [21, 5, 30, 10] , "Price": [200, 300, 40, 250] } let df = new dfd.DataFrame(data) let sub_df = df.iloc({rows: ["1:3"]}) sub_df.print() Copy ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Name │ Count │ Price ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ Mango │ 5 │ 300 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ Banana │ 30 │ 40 ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ By lists of integer position locations: Copy const dfd = require("danfojs-node") let data = { "Name": ["Apples", "Mango", "Banana", "Pear"] , "Count": [21, 5, 30, 10] , "Price": [200, 300, 40, 250] } let df = new dfd.DataFrame(data) let sub_df = df.iloc({rows: [1,3], columns: [0,2]}) sub_df.print() Copy ╔═══╤═══════════════════╤═══════════════════╗ ║ │ Name │ Price ║ ╟───┼───────────────────┼───────────────────╢ ║ 1 │ Mango │ 300 ║ ╟───┼───────────────────┼───────────────────╢ ║ 3 │ Pear │ 250 ║ ╚═══╧═══════════════════╧═══════════════════╝ For slicing rows explicitly: Copy const dfd = require("danfojs-node") let data = { "Name": ["Apples", "Mango", "Banana", "Pear"] , "Count": [21, 5, 30, 10] , "Price": [200, 300, 40, 250] } let df = new dfd.DataFrame(data) let sub_df = df.iloc({rows: ["2:3"], columns: [":"]}) sub_df.print() Copy ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Name │ Count │ Price ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ Banana │ 30 │ 40 ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ For slicing columns explicitly: Copy const dfd = require("danfojs-node") let data = { "Name": ["Apples", "Mango", "Banana", "Pear"] , "Count": [21, 5, 30, 10] , "Price": [200, 300, 40, 250] } let df = new dfd.DataFrame(data) let sub_df = df.iloc({rows: [":"], columns: ["1:2"]}) sub_df.print() Copy ╔════════════╤═══════════════════╗ ║ │ Count ║ ╟────────────┼───────────────────╢ ║ 0 │ 21 ║ ╟────────────┼───────────────────╢ ║ 1 │ 5 ║ ╟────────────┼───────────────────╢ ║ 2 │ 30 ║ ╟────────────┼───────────────────╢ ║ 3 │ 10 ║ ╚════════════╧═══════════════════╝ #### [](#selection-with-boolean-mask) Selection with Boolean Mask You can select subsections from a DataFrame by a booelan condition mask. E.g. In the following code, we select and return only rows where the column `Count` is greater than 10. Copy let data = { "Name": ["Apples", "Mango", "Banana", "Pear"], "Count": [21, 5, 30, 10], "Price": [200, 300, 40, 250] } let df = new dfd.DataFrame(data) let sub_df = df.iloc({ rows: df["Count"].gt(10) }) sub_df.print() Copy //output ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Name │ Count │ Price ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ Apples │ 21 │ 200 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ Banana │ 30 │ 40 ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ A Boolean mask for filtering also works for multiple conditions using `and` & `or` functions. E.g, In the following code, we select and return only rows where the column `Count` is greater than 10 and column `Name` is equal to `Apples`. Copy let sub_df = df.iloc({ rows: df["Count"].gt(10).and(df["Name"].eq("Apples")), columns: [0] }) sub_df.print() //output ╔════════════╤═══════════════════╗ ║ │ Name ║ ╟────────────┼───────────────────╢ ║ 0 │ Apples ║ ╚════════════╧═══════════════════╝ #### [](#boolean-querying-filtering) Boolean Querying/Filtering The best way to query data is to use a boolean mask just as we demonstrated above with iloc and loc. For example, in the following code, we use a condition parameter to query the DataFrame: Copy let data = { "A": ["Ng", "Yu", "Mo", "Ng"], "B": [34, 4, 5, 6], "C": [20, 20, 30, 40] } let df = new dfd.DataFrame(data) let query_df = df.query(df["B"].gt(5)) query_df.print() Copy ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ Ng │ 34 │ 20 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ Ng │ 6 │ 40 ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ Querying by a boolean condition is supported from v0.3.0 and above. It also supports condition chaining as long as the final boolean mask is the same lenght as the DataFrame rows. For example in the following code, we use multiple chaining conditions: Copy let data = { "A": ["Ng", "Yu", "Mo", "Ng"], "B": [34, 4, 5, 6], "C": [20, 20, 30, 40] } let query_df = df.query( df["B"].gt(5).and(df["C"].lt(0))) query_df.print() //after query //output ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ Ng │ 34 │ 20 ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ #### [](#adding-a-new-column) Adding a new column Setting a new column automatically aligns the data by the indexes. NodeBrowser Copy const dfd = require("danfojs-node") let data = { "A": [30, 1, 2, 3] , "B": [34, 4, 5, 6] , "C": [20, 20, 30, 40] } let df = new dfd.DataFrame(data) df.print() let new_col = [1, 2, 3, 4] df.addColumn("D", new_col, { inplace: true }); //happens inplace df.print() Copy Document Copy //before adding column ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ 30 │ 34 │ 20 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ 1 │ 4 │ 20 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ 2 │ 5 │ 30 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ 3 │ 6 │ 40 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╝ //after adding column Shape: (4,3) ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C │ D ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ 1 │ 2 │ 3 │ 25 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ 4 │ 5 │ 6 │ 35 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ 20 │ 30 │ 40 │ 45 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ 39 │ 89 │ 78 │ 55 ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ ### [](#missing-data) Missing data **NaN, null,** and **undefined** represent missing data in Danfo.js. These values can be dropped or filled using some functions available in Danfo.js. To drop any columns that have missing data: NodeBrowser Copy const dfd = require("danfojs-node") let data = [[1, 2, 3], [NaN, 5, 6], [NaN, 30, 40], [39, 20, 78]] let cols = ["A", "B", "C"] let df = new dfd.DataFrame(data, { columns: cols }) df.print() let df_drop = df.dropNa({ axis: 0 }) df_drop.print() Copy Document Copy //Before dropping ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ 1 │ 2 │ 3 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ NaN │ 5 │ 6 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ NaN │ 30 │ 40 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ 39 │ 20 │ 78 ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ //after dropping ╔════════════╤═══════════════════╤═══════════════════╗ ║ │ B │ C ║ ╟────────────┼───────────────────┼───────────────────╢ ║ 0 │ 2 │ 3 ║ ╟────────────┼───────────────────┼───────────────────╢ ║ 1 │ 5 │ 6 ║ ╟────────────┼───────────────────┼───────────────────╢ ║ 2 │ 30 │ 40 ║ ╟────────────┼───────────────────┼───────────────────╢ ║ 3 │ 20 │ 78 ║ ╚════════════╧═══════════════════╧═══════════════════╝ To drop row(s) with have missing data, set the axis to 1: Copy const dfd = require("danfojs-node") let data = [[1, 2, 3], [NaN, 5, 6], [20, 30, 40], [39, 34, 78]] let cols = ["A", "B", "C"] let df = new dfd.DataFrame(data, { columns: cols }) df.print() let df_drop = df.dropNa({ axis: 1 }) df_drop.print() Copy //Before dropping ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ 1 │ 2 │ 3 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ NaN │ 5 │ 6 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ 20 │ 30 │ 40 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ 39 │ 34 │ 78 ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ //after dropping ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ 1 │ 2 │ 3 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ 39 │ 20 │ 78 ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ Filling missing data: Copy const dfd = require("danfojs-node") let data = { "Name": ["Apples", "Mango", "Banana", NaN], "Count": [NaN, 5, NaN, 10], "Price": [200, 300, 40, 250] } let df = new dfd.DataFrame(data) let df_filled = df.fillNa("Apples") df_filled.print() Copy ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Name │ Count │ Price ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ Apples │ Apples │ 200 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ Mango │ 5 │ 300 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ Banana │ Apples │ 40 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ Apples │ 10 │ 250 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╝ Filling missing values in specific columns with specific values: Copy const dfd = require("danfojs-node") let data = { "Name": ["Apples", "Mango", "Banana", NaN], "Count": [NaN, 5, NaN, 10], "Price": [200, 300, 40, 250] } let df = new dfd.DataFrame(data) df.print() let df_filled = df.fillNa(["Apples", df["Count"].mean()], { columns: ["Name", "Count"] }) df_filled.print() Copy ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Name │ Count │ Price ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ Apples │ 7.5 │ 200 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ Mango │ 5 │ 300 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ Banana │ 7.5 │ 40 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ Apples │ 10 │ 250 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╝ To get the boolean mask where values are `nan`. Copy const dfd = require("danfojs-node") let data = {"Name":["Apples", "Mango", "Banana", undefined], "Count": [NaN, 5, NaN, 10], "Price": [200, 300, 40, 250]} let df = new dfd.DataFrame(data) df.isNa().print() Copy ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Name │ Count │ Price ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ false │ true │ false ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ false │ false │ false ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ false │ true │ false ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ true │ false │ false ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╝ ### [](#operations) Operations #### [](#stats) Stats Operations, in general, _exclude_ missing data. Performing a descriptive statistic: NodeBrowser Copy const dfd = require("danfojs-node") data = [[11, 20, 3], [1, 15, 6], [2, 30, 40], [2, 89, 78]] cols = ["A", "B", "C"] let df = new dfd.DataFrame(data, { columns: cols }) df.print() df.mean().print() //defaults to column (1) axis Copy Document Copy ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ 0 │ 1 │ 2 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ 11 │ 20 │ 3 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ 1 │ 15 │ 6 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ 2 │ 30 │ 40 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ 2 │ 89 │ 78 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╝ ╔═══╤════════════════════╗ ║ 0 │ 11.333333333333334 ║ ╟───┼────────────────────╢ ║ 1 │ 7.333333333333333 ║ ╟───┼────────────────────╢ ║ 2 │ 24 ║ ╟───┼────────────────────╢ ║ 3 │ 56.333333333333336 ║ ╚═══╧════════════════════╝ Same operation on the row axis: Copy const dfd = require("danfojs-node") data = [[11, 20, 3], [1, 15, 6], [2, 30, 40], [2, 89, 78]] cols = ["A", "B", "C"] let df = new dfd.DataFrame(data) df.print() df.mean({ axis: 0 }).print() //row axis=0, column=1 Copy ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ 0 │ 1 │ 2 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ 11 │ 20 │ 3 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ 1 │ 15 │ 6 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ 2 │ 30 │ 40 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ 2 │ 89 │ 78 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╝ ╔═══╤═══════╗ ║ A │ 4 ║ ╟───┼───────╢ ║ B │ 38.5 ║ ╟───┼───────╢ ║ C │ 31.75 ║ ╚═══╧═══════╝ Operations on objects with different dimensionality and need alignment. Danfo automatically broadcasts along the specified dimension. Copy const dfd = require("danfojs-node") let data = { "Col1": [1, 4, 5, 1], "Col2": [3, 2, 0, 4] } let df = new dfd.DataFrame(data) let sf = new dfd.Series([4, 5]) let df_new = df.sub(sf, { axis: 1 }) df_new.print() Copy ╔═══╤═══════════════════╤═══════════════════╗ ║ │ Col1 │ Col2 ║ ╟───┼───────────────────┼───────────────────╢ ║ 0 │ -3 │ -2 ║ ╟───┼───────────────────┼───────────────────╢ ║ 1 │ 0 │ -3 ║ ╟───┼───────────────────┼───────────────────╢ ║ 2 │ 1 │ -5 ║ ╟───┼───────────────────┼───────────────────╢ ║ 3 │ -3 │ -1 ║ ╚═══╧═══════════════════╧═══════════════════╝ #### [](#apply) Apply Applying functions to the data along a specified axis. If axis = 1 (default), then the specified function (`callable)` will be called with each row data, and vice versa: Copy const dfd = require("danfojs") let data = [[1, 2, 3], [4, 5, 6], [20, 30, 40], [39, 89, 78]] let cols = ["A", "B", "C"] let df = new dfd.DataFrame(data, { columns: cols }) function sum_vals(col) { return col.reduce((a, b) => a + b, 0); } let df_new = df.apply(sum_vals, { axis: 1 }) df_new.print() Copy //before applying ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ 1 │ 2 │ 3 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ 4 │ 5 │ 6 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ 20 │ 30 │ 40 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ 39 │ 89 │ 78 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╝ //after applying ╔═══╤═════╗ ║ 0 │ 6 ║ ╟───┼─────╢ ║ 1 │ 15 ║ ╟───┼─────╢ ║ 2 │ 90 ║ ╟───┼─────╢ ║ 3 │ 206 ║ ╚═══╧═════╝ Applying Element wise operations to the data: You can use the `applyMap` function if you need to apply a function to each element in the DataFrame. `applyMap` works element-wise. Copy const dfd = require("danfojs-node") let data = [[1, 2, 3], [4, 5, 6], [20, 30, 40], [39, 89, 78]] let cols = ["A", "B", "C"] let df = new dfd.DataFrame(data, { columns: cols }) function sum_vals(x) { return x + 10 } let df_new = df.applyMap(sum_vals) df_new.print() Copy //before applying ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ 1 │ 2 │ 3 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ 4 │ 5 │ 6 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ 20 │ 30 │ 40 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ 39 │ 89 │ 78 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╝ //after applyMap ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ 11 │ 12 │ 13 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ 14 │ 15 │ 16 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ 30 │ 40 │ 50 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ 49 │ 99 │ 88 ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ #### [](#string-methods) String Methods Series is equipped with a set of string processing methods in the **str** attribute that make it easy to operate on each element of the array, as in the code snippet below. Note that pattern-matching in **str** generally uses JavaScript [regular expressions](https://docs.python.org/3/library/re.html) by default (and in some cases always uses them). Copy const dfd = require("danfojs-node") let s = new dfd.Series(['A', 'B', 'C', 'Aaba', 'Baca', 'CABA', 'dog', 'cat']) let lower_s = s.str.toLowerCase() lower_s.print() Copy ╔═══╤══════════════════════╗ ║ │ 0 ║ ╟───┼──────────────────────╢ ║ 0 │ a ║ ╟───┼──────────────────────╢ ║ 1 │ b ║ ╟───┼──────────────────────╢ ║ 2 │ c ║ ╟───┼──────────────────────╢ ║ 3 │ aaba ║ ╟───┼──────────────────────╢ ║ 4 │ baca ║ ╟───┼──────────────────────╢ ║ 5 │ caba ║ ╟───┼──────────────────────╢ ║ 6 │ dog ║ ╟───┼──────────────────────╢ ║ 7 │ cat ║ ╚═══╧══════════════════════╝ See more string [accessors](https://jsdata.gitbook.io/danfojs/api-reference/series#accessors) here ### [](#merge) Merge #### [](#concat) Concat danfo provides various methods for easily combining together Series and DataFrame objects with various kinds of set logic for the indexes and relational algebra functionality in the case of join / merge-type operations. Concatenating DataFrame together with [`concat()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.concat.html#pandas.concat) : Copy const dfd = require("danfojs-node") let data = [['K0', 'k0', 'A0', 'B0'], ['k0', 'K1', 'A1', 'B1'],\ ['K1', 'K0', 'A2', 'B2'], ['K2', 'K2', 'A3', 'B3']] let data2 = [['K0', 'k0', 'C0', 'D0'], ['K1', 'K0', 'C1', 'D1'],\ ['K1', 'K0', 'C2', 'D2'], ['K2', 'K0', 'C3', 'D3']] let colum1 = ['Key1', 'Key2', 'A', 'B'] let colum2 = ['Key1', 'Key2', 'A', 'D'] let df1 = new dfd.DataFrame(data, { columns: colum1 }) let df2 = new dfd.DataFrame(data2, { columns: colum2 }) let com_df = dfd.concat({ dfList: [df1, df2], axis: 1 }) //along column axis com_df.print() Copy ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Key1 │ Key2 │ A │ B │ Key11 │ Key21 │ A1 │ D ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ K0 │ k0 │ A0 │ B0 │ K0 │ k0 │ C0 │ D0 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ k0 │ K1 │ A1 │ B1 │ K1 │ K0 │ C1 │ D1 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ K1 │ K0 │ A2 │ B2 │ K1 │ K0 │ C2 │ D2 ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ K2 │ K2 │ A3 │ B3 │ K2 │ K0 │ C3 │ D3 ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ Concatenate along row axis (0). Copy const dfd = require("danfojs-node") let data = [['K0', 'k0', 'A0', 'B0'], ['k0', 'K1', 'A1', 'B1'],\ ['K1', 'K0', 'A2', 'B2'], ['K2', 'K2', 'A3', 'B3']] let data2 = [['K0', 'k0', 'C0', 'D0'], ['K1', 'K0', 'C1', 'D1'],\ ['K1', 'K0', 'C2', 'D2'], ['K2', 'K0', 'C3', 'D3']] let colum1 = ['Key1', 'Key2', 'A', 'B'] let colum2 = ['Key1', 'Key2', 'A', 'D'] let df1 = new dfd.DataFrame(data, { columns: colum1 }) let df2 = new dfd.DataFrame(data2, { columns: colum2 }) let com_df = dfd.concat({ dfList: [df1, df2], axis: 0 }) //along row axis com_df.print() Copy ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Key1 │ Key2 │ A │ B │ D ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ K0 │ k0 │ A0 │ B0 │ NaN ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ k0 │ K1 │ A1 │ B1 │ NaN ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ K1 │ K0 │ A2 │ B2 │ NaN ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ K2 │ K2 │ A3 │ B3 │ NaN ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 4 │ K0 │ k0 │ C0 │ NaN │ D0 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 5 │ K1 │ K0 │ C1 │ NaN │ D1 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 6 │ K1 │ K0 │ C2 │ NaN │ D2 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 7 │ K2 │ K0 │ C3 │ NaN │ D3 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ #### [](#join) Join SQL style merges. See the Pandas [Database style joining](https://pandas.pydata.org/pandas-docs/stable/user_guide/merging.html#merging-join) section for more info. Copy const dfd = require("danfojs-node") let data = [['K0', 'k0', 'A0', 'B0'], ['k0', 'K1', 'A1', 'B1'],\ ['K1', 'K0', 'A2', 'B2'], ['K2', 'K2', 'A3', 'B3']] let data2 = [['K0', 'k0', 'C0', 'D0'], ['K1', 'K0', 'C1', 'D1'],\ ['K1', 'K0', 'C2', 'D2'], ['K2', 'K0', 'C3', 'D3']] let colum1 = ['Key1', 'Key2', 'A', 'B'] let colum2 = ['Key1', 'Key2', 'A', 'D'] let df1 = new dfd.DataFrame(data, { columns: colum1 }) let df2 = new dfd.DataFrame(data2, { columns: colum2 }) df1.print() df2.print() let merge_df = dfd.merge({ "left": df1, "right": df2, "on": ["Key1"], how: "inner" }) merge_df.print() Copy //first DataFrame ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Key1 │ Key2 │ A │ B ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ K0 │ k0 │ A0 │ B0 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ k0 │ K1 │ A1 │ B1 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ K1 │ K0 │ A2 │ B2 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ K2 │ K2 │ A3 │ B3 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ //Second DataFrame ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Key1 │ Key2 │ A │ D ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ K0 │ k0 │ C0 │ D0 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ K1 │ K0 │ C1 │ D1 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ K1 │ K0 │ C2 │ D2 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ K2 │ K0 │ C3 │ D3 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ //After inner join on column 'Key1' ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Key1 │ Key2 │ A │ B │ Key2_1 │ A_1 │ D ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ K0 │ k0 │ A0 │ B0 │ k0 │ C0 │ D0 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ K1 │ K0 │ A2 │ B2 │ K0 │ C1 │ D1 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ K1 │ K0 │ A2 │ B2 │ K0 │ C2 │ D2 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ K2 │ K2 │ A3 │ B3 │ K0 │ C3 │ D3 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ See the [merge](https://jsdata.gitbook.io/danfojs/api-reference/general-functions/danfo.merge) section for more examples ### [](#grouping) Grouping By “group by” we are referring to a process involving one or more of the following steps: > * **Splitting** the data into groups based on some criteria > > * **Applying** a function to each group independently > > * **Combining** the results into a data structure > See the [Grouping section](/api-reference/groupby) . Copy const dfd = require("danfojs-node") let data ={'A': ['foo', 'bar', 'foo', 'bar',\ 'foo', 'bar', 'foo', 'foo'], 'B': ['one', 'one', 'two', 'three',\ 'two', 'two', 'one', 'three'], 'C': [1,3,2,4,5,2,6,7], 'D': [3,2,4,1,5,6,7,8] } let df = new dfd.DataFrame(data) let grp = df.groupby(["A"]) grp.get_groups(["foo"]).print() grp.get_groups(["bar"]).print() Copy ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C │ D ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ foo │ one │ 1 │ 3 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ foo │ two │ 2 │ 4 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ foo │ two │ 5 │ 5 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ foo │ one │ 6 │ 7 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 4 │ foo │ three │ 7 │ 8 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ Shape: (3,4) ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C │ D ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ bar │ one │ 3 │ 2 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ bar │ three │ 4 │ 1 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ bar │ two │ 2 │ 6 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ Grouping and then applying the[`sum()`](/api-reference/groupby/groupby.sum) function to the resulting groups. Copy const dfd = require("danfojs-node") let data = { A: ["foo", "bar", "foo", "bar", "foo", "bar", "foo", "foo"], B: ["one", "one", "two", "three", "two", "two", "one", "three"], C: [1, 3, 2, 4, 5, 2, 6, 7], D: [3, 2, 4, 1, 5, 6, 7, 8], }; let df = new dfd.DataFrame(data); let grp = df.groupby(["A"]); grp.col(["C"]).sum().print(); Copy ╔═══╤═══════════════════╤═══════════════════╗ ║ │ A │ C_sum ║ ╟───┼───────────────────┼───────────────────╢ ║ 0 │ foo │ 21 ║ ╟───┼───────────────────┼───────────────────╢ ║ 1 │ bar │ 9 ║ ╚═══╧═══════════════════╧═══════════════════╝ Grouping by multiple columns forms a hierarchical index, and again we can apply the[`sum()`](/api-reference/groupby/groupby.sum) function. Copy const dfd = require("danfojs-node") let data ={'A': ['foo', 'bar', 'foo', 'bar',\ 'foo', 'bar', 'foo', 'foo'], 'B': ['one', 'one', 'two', 'three',\ 'two', 'two', 'one', 'three'], 'C': [1,3,2,4,5,2,6,7], 'D': [3,2,4,1,5,6,7,8] } let df = new dfd.DataFrame(data) let grp = df.groupby(["A","B"]) grp.col(["C"]).sum().print() Copy ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ A │ B │ C_sum ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ foo │ one │ 7 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ foo │ two │ 7 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ foo │ three │ 7 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 3 │ bar │ one │ 3 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 4 │ bar │ two │ 2 ║ ╟───┼───────────────────┼───────────────────┼───────────────────╢ ║ 5 │ bar │ three │ 4 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╝ ### [](#time-series) Time series danfo provides a simple but powerful, and efficient functionality for working with DateTime data. See the **dt** [Accessors](https://jsdata.gitbook.io/danfojs/api-reference/series#accessors) section. Copy const dfd = require("danfojs-node") let data = new dfd.dateRange({"start":'2018-01', freq:'M', period:3}) let sf = new dfd.Series(data) //print series sf.print() //print month names sf.dt.monthName().print() Copy ╔═══╤══════════════════════╗ ║ │ 0 ║ ╟───┼──────────────────────╢ ║ 0 │ 1/1/2018, 1:00:00 AM ║ ╟───┼──────────────────────╢ ║ 1 │ 2/1/2018, 1:00:00 AM ║ ╟───┼──────────────────────╢ ║ 2 │ 3/1/2018, 1:00:00 AM ║ ╚═══╧══════════════════════╝ ╔═══╤══════════╗ ║ 0 │ January ║ ╟───┼──────────╢ ║ 1 │ February ║ ╟───┼──────────╢ ║ 2 │ March ║ ╚═══╧══════════╝ More Examples: Copy const dfd = require("danfojs-node") let data = new dfd.dateRange({"start":'2018-01', freq:'M', period:3}) let sf = new dfd.Series(data) //print series sf.print() //print week day names sf.dt.dayOfWeekName().print() Copy ╔═══╤══════════════════════╗ ║ │ 0 ║ ╟───┼──────────────────────╢ ║ 0 │ 1/1/2018, 1:00:00 AM ║ ╟───┼──────────────────────╢ ║ 1 │ 2/1/2018, 1:00:00 AM ║ ╟───┼──────────────────────╢ ║ 2 │ 3/1/2018, 1:00:00 AM ║ ╚═══╧══════════════════════╝ ╔═══╤══════════╗ ║ 0 │ Monday ║ ╟───┼──────────╢ ║ 1 │ Thursday ║ ╟───┼──────────╢ ║ 2 │ Thursday ║ ╚═══╧══════════╝ ### [](#plotting) Plotting See the [Plotting](/api-reference/plotting) docs. We currently support [Plotly.js](https://plotly.com/javascript/) for plotting. In the future, we plan other JS plotting libraries like Vega, D3. Using the `plot` API, you can make interactive plots from DataFrame and Series. Plotting only works in the browser/client-side version of Danfo.js, and requires an HTML div to display plots. Copy Document
On a DataFrame, the [`plot()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.plot.html#pandas.DataFrame.plot) method exposes various [plot types](/api-reference/plotting) . And by default, all columns are plotted unless specified otherwise. Copy Document
### [](#getting-data-in-out) Getting data in/out #### [](#csv) CSV [Writing to a CSV file.](/api-reference/dataframe/dataframe.to_csv) Convert any DataFrame to csv format. In NodeJs, if a file path is specified, then the CSV is saved to the path, else it is returned as a string. In the browser, you can automatically download the file as CSV by setting the `download` paramater to `true`. Copy const dfd = require("danfojs-node") let data = { "Abs": [20.2, 30, 47.3], "Count": [34, 4, 5], "country code": ["NG", "FR", "GH"] } let df = new dfd.DataFrame(data) const csv = dfd.toCSV(df) console.log(csv); //output Abs,Count,country code 20.2,34,NG 30,4,FR 47.3,5,GH dfd.toCSV(df, {filePath: "testOut.csv" }) //writes to file system in Nodejs dfd.toCSV(df, {fileName: "testOut", download: true }) //downloads the file in browser version Copy Abs,Count,country code 20.2,34,NG 30,4,FR 47.3,5,GH [Reading from a CSV file.](https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html#io-read-csv-table) The **readCSV** method can read CSV files from local disk, or over the internet. Both full and relative paths are supported. For example, to read a CSV file at the path **/home/Desktop/titanic.csv**, you can do the following: JavaScriptBrowser Copy const dfd = require("danfojs") dfd.readCSV("/home/Desktop/titanic.csv") .then(df => { //do something with the CSV file df.head().print() }).catch(err=>{ console.log(err); }) Copy Document #### [](#json) JSON Writing to [JSON](/api-reference/dataframe/dataframe.to_json) format Copy const dfd = require("danfojs-node") let data = { "Abs": [20.2, 30, 47.3], "Count": [34, 4, 5], "country code": ["NG", "FR", "GH"] } let df = new dfd.DataFrame(data) const json = dfd.toJSON(df) console.log(json); //output [\ { Abs: 20.2, Count: 34, 'country code': 'NG' },\ { Abs: 30, Count: 4, 'country code': 'FR' },\ { Abs: 47.3, Count: 5, 'country code': 'GH' }\ ] const json = dfd.toJSON(df, {format: "row"}) console.log(json); //output { Abs: [ 20.2, 30, 47.3 ], Count: [ 34, 4, 5 ], 'country code': [ 'NG', 'FR', 'GH' ] } [PreviousDanfo.js Documentation](/) [NextAPI reference](/api-reference) Last updated 2 years ago Was this helpful? ![](https://danfo.jsdata.org/~gitbook/image?url=https%3A%2F%2F3251798050-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-MB05JeHIrR5pfcM-_7L%252Fuploads%252Fgit-blob-a884a2060a233ec9517e12facbe7dc6b3937374e%252Fnewplot-29-%2520%282%29.png%3Falt%3Dmedia%26token%3D51ac824e-0d1d-48b1-81ae-4d0960247905&width=768&dpr=4&quality=100&sign=4d3780b&sv=2) ![](https://danfo.jsdata.org/~gitbook/image?url=https%3A%2F%2F3251798050-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F-MB05JeHIrR5pfcM-_7L%252Fuploads%252Fgit-blob-9f34611dd09a280c06d1a1f30b28dafc5f3936cb%252Fnewplot-2-%2520%281%29%2520%281%29.png%3Falt%3Dmedia%26token%3De71cca54-8a24-48db-85d0-6a2543425774&width=768&dpr=4&quality=100&sign=eb680ae8&sv=2) --- # danfo.Str | Danfo.js For example, in the following example, we convert a Series to an `Str` instance and apply a couple of **String** methods. Node Copy import { Str, Series } from "danfojs-node" const sf = new Series(["Dog", "Cat", "Bird", "Fish", "ShArk", "tiGer"]) const sfStr = new Str(sf) sfStr.toLowerCase().print() sfStr.toUpperCase().print() sfStr.join("Added", "-").print() Copy // output ╔═══╤═══════╗ ║ 0 │ dog ║ ╟───┼───────╢ ║ 1 │ cat ║ ╟───┼───────╢ ║ 2 │ bird ║ ╟───┼───────╢ ║ 3 │ fish ║ ╟───┼───────╢ ║ 4 │ shark ║ ╟───┼───────╢ ║ 5 │ tiger ║ ╚═══╧═══════╝ ╔═══╤═══════╗ ║ 0 │ DOG ║ ╟───┼───────╢ ║ 1 │ CAT ║ ╟───┼───────╢ ║ 2 │ BIRD ║ ╟───┼───────╢ ║ 3 │ FISH ║ ╟───┼───────╢ ║ 4 │ SHARK ║ ╟───┼───────╢ ║ 5 │ TIGER ║ ╚═══╧═══════╝ ╔═══╤═════════════╗ ║ 0 │ Dog-Added ║ ╟───┼─────────────╢ ║ 1 │ Cat-Added ║ ╟───┼─────────────╢ ║ 2 │ Bird-Added ║ ╟───┼─────────────╢ ║ 3 │ Fish-Added ║ ╟───┼─────────────╢ ║ 4 │ ShArk-Added ║ ╟───┼─────────────╢ ║ 5 │ tiGer-Added ║ ╚═══╧═════════════╝ [Previousdanfo.Utils](/api-reference/general-functions/danfo.utils) [Nextdanfo.Dt](/api-reference/general-functions/danfo.dt) Last updated 3 years ago Was this helpful? --- # danfo.streamJSON | Danfo.js danfo.**streamJSON**(filePath, callback, options) Parameters Type Description filePath string URL or local file path to CSV file. callback Function Callback function to be called once the specifed rows are parsed into DataFrame. options object Optional configuration object. We use the `request` library for reading remote json files, Hence all `request` parameters such as `method`, `headers`, are supported. The **streamJSON** function streams a JSON file from a local or remote location in chunks. Each intermediate chunk is passed as a DataFrame to the callback function. [](#stream-json-file-from-local-path) **Stream JSON file from local path** ------------------------------------------------------------------------------- Node Copy const dfd = require("danfojs-node") const path = require("path") const filePath = path.join(process.cwd(), "raw_data", "book_small.json"); dfd.streamJSON(filePath, (df) => { if (df) { // Do any processing here df.print(); } }); Output Copy //Showing the last rows ... ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ book_id │ title │ image_url │ authors ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 10 │ 32848471 │ Egomaniac │ https://images.… │ Vi Keeland ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ book_id │ title │ image_url │ authors ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 11 │ 33288638 │ Wait for It │ https://s.gr-as… │ Mariana Zapata ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ [](#stream-json-file-from-remote-path) **Stream JSON file from remote path** --------------------------------------------------------------------------------- Node Copy const dfd = require("danfojs-node") const path = require("path") const remoteFile = "https://raw.githubusercontent.com/opensource9ja/danfojs/dev/danfojs-node/tests/samples/book.json" const callback = (df) => { //Perform any processing here if (df) { df.print(); } } dfd.streamJSON(remoteFile, callback, { header: true }) Output Copy //Showing a few rows ... ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ book_id │ title │ image_url │ authors ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 10 │ 32848471 │ Egomaniac │ https://images.… │ Vi Keeland ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ book_id │ title │ image_url │ authors ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 11 │ 33288638 │ Wait for It │ https://s.gr-as… │ Mariana Zapata ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ [Previousdanfo.streamCsvTransformer](/api-reference/general-functions/danfo.streamcsvtransformer) [Nextdanfo.streamCSV](/api-reference/general-functions/danfo.streamcsv) Last updated 3 years ago Was this helpful? --- # danfo.tensorflow | Danfo.js danfo.**tensorflow** **Returns:** > return [Tensorflow.js](https://www.npmjs.com/package/@tensorflow/tfjs) > library [](#examples) **Examples** ------------------------------- NodeBrowser Copy const dfd = require("danfojs-node") const tf = dfd.tensorflow let tensor_arr = tf.tensor2d([[12, 34, 2.2, 2], [30, 30, 2.1, 7]]) console.log(tensor_arr) Copy Document Output Copy Tensor { kept: false, isDisposedInternal: false, shape: [ 2, 4 ], dtype: 'float32', size: 8, strides: [ 4 ], dataId: {}, id: 4, rankType: '2' } [PreviousGeneral Functions](/api-reference/general-functions) [Nextdanfo. convertFunctionTotransformer](/api-reference/general-functions/danfo.-convertfunctiontotransformer) Last updated 3 years ago Was this helpful? --- # API reference | Danfo.js A stable version of Danfojs (v1), has been released, and it comes with full Typescript support, new features, and many bug fixes. See release note [here](/release-notes#latest-release-node-v1.0.0-browser-v1.0.0) . There are a couple of breaking changes, so we have prepared a short migration [guide](/examples/migrating-to-the-stable-version-of-danfo.js) for pre-v1 users. * [General Functions](/api-reference/general-functions) * [Data manipulations](/api-reference/general-functions#data-manipulations) * [Data Processing/Normalization](/api-reference/general-functions#data-processing-normalization) * [Top-level dealing with datetime like](/api-reference/general-functions#top-level-dealing-with-datetime) * [Input/output](/api-reference/input-output) * [CSV](/api-reference/input-output#csv) * [JSON](/api-reference/input-output#json) * [Series](/api-reference/series) * [Attributes](/api-reference/series#attributes) * [Conversion](/api-reference/series#conversion) * [Indexing, iteration](/api-reference/series#indexing-iteration) * [Binary operator functions](/api-reference/series#binary-operator-functions) * [Function application, GroupBy & window](/api-reference/series#function-application-and-groupby) * [Computations / descriptive stats](/api-reference/series#computations-descriptive-stats) * [Reindexing / selection / label manipulation](/api-reference/series#reindexing-selection-label-manipulation) * [Missing data handling](/api-reference/series#missing-data-handling) * [Reshaping, sorting](/api-reference/series#reshaping-sorting) * [Accessors](/api-reference/series#accessors) * [Serialization / IO / conversion](/api-reference/series#serialization-io-conversion) * [DataFrame](/api-reference/dataframe) * [Attributes](/api-reference/dataframe#attributes) * [Conversion](/api-reference/dataframe#conversion) * [Indexing, iteration](/api-reference/dataframe#indexing-iteration) * [Binary operator functions](/api-reference/dataframe#binary-operator-functions) * [Function application, GroupBy & window](/api-reference/dataframe#function-application-and-groupby) * [Computations / descriptive stats](/api-reference/dataframe#computations-descriptive-stats) * [Reindexing / selection / label manipulation](/api-reference/dataframe#reindexing-selection-label-manipulation) * [Missing data handling](/api-reference/dataframe#missing-data-handling) * [Reshaping, sorting, transposing](/api-reference/dataframe#sorting-and-transposing) * [Combining / comparing / joining / merging](/api-reference/dataframe#combining-comparing-joining-merging) * [Serialization / IO / conversion](/api-reference/dataframe#serialization-io-conversion) * [Plotting](/api-reference/plotting) * [Line Charts](/api-reference/plotting/line-charts) * [Bar Charts](/api-reference/plotting/bar-charts) * [Scatter Plots](/api-reference/plotting/scatter-plots) * [Histograms](/api-reference/plotting/histograms) * [Pie Charts](/api-reference/plotting/pie-charts) * [Tables](/api-reference/plotting/tables) * [Box Plots](/api-reference/plotting/box-plots) * [Violin Plots](/api-reference/plotting/violin-plots) * [Timeseries Plots](/api-reference/plotting/timeseries-plots) * [GroupBy](https://pandas.pydata.org/pandas-docs/stable/reference/groupby.html) * [Indexing, iteration](/api-reference/groupby#indexing-iteration) * [Function application](/api-reference/groupby#function-application) * [Computations / descriptive stats](/api-reference/groupby#computations-descriptive-stats) [PreviousGetting Started](/getting-started) [NextGeneral Functions](/api-reference/general-functions) Last updated 3 years ago Was this helpful? --- # danfo.streamCSV | Danfo.js danfo.**streamCSV**(filePath, callback, options) Parameters Type Description filePath string URL or local file path to CSV file. callback Function Callback function to be called once the specifed rows are parsed into DataFrame. options object The **streamCSV** function streams a CSV file from a local or remote location in chunks. Each intermediate chunk is passed as a DataFrame to the callback function. [](#stream-csv-file-from-local-path) **Stream CSV file from local path** ----------------------------------------------------------------------------- Node Copy const dfd = require("danfojs-node") const path = require("path") const filePath = path.join(process.cwd(), "raw_data", "titanic.csv"); dfd.streamCSV(filePath, (df) => { if (df) { // Do any processing here df.print(); } }); Output Copy //Showing few rows ... ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ PassengerId │ Survived │ Pclass │ Name │ ... │ Fare │ Cabin │ Embarked ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 676 │ 687 │ 0 │ 3 │ Panula, Mr. Jaa… │ ... │ 39.6875 │ │ S ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ PassengerId │ Survived │ Pclass │ Name │ ... │ Fare │ Cabin │ Embarked ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 677 │ 688 │ 0 │ 3 │ Dakic, Mr. Bran… │ ... │ 10.1708 │ │ S ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ ... [](#stream-csv-file-from-remote-path) **Stream CSV file from remote path** ------------------------------------------------------------------------------- Node Copy const dfd = require("danfojs-node") const remoteFile = "https://raw.githubusercontent.com/opensource9ja/danfojs/dev/danfojs-node/tests/samples/titanic.csv" const callback = (df) => { //Perform any processing here if (df) { df.print(); } } dfd.streamCSV(remoteFile, callback, { header: true }) Output Copy //Showing a few rows ... ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Survived │ Pclass │ Name │ Sex │ Age │ Siblings/Spouse… │ Parents/Childre… │ Fare ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 523 │ 0 │ 1 │ Mr. John Farthi… │ male │ 49 │ 0 │ 0 │ 221.7792 ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Survived │ Pclass │ Name │ Sex │ Age │ Siblings/Spouse… │ Parents/Childre… │ Fare ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 524 │ 0 │ 3 │ Mr. Johan Werne… │ male │ 39 │ 0 │ 0 │ 7.925 ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ ... [Previousdanfo.streamJSON](/api-reference/general-functions/danfo.streamjson) [Nextdanfo.Utils](/api-reference/general-functions/danfo.utils) Last updated 3 years ago Was this helpful? Optional configuration object. Supports all config options. [Papaparse](https://www.papaparse.com/docs#config) --- # danfo.streamCsvTransformer | Danfo.js danfo.**streamCsvTransformer**(func) Parameters Type Description inputFilePath Function The path to the CSV file to stream from. transformer Function The transformer function to apply to each row. Note that each row of the CSV file is passed as a DataFrame with a single row to the transformer function, and the transformer function is expected to return a transformed DataFrame. options object Configuration options for the pipeline. These include: * `outputFilePath` The local file path to write the transformed CSV file to. * `customCSVStreamWriter` A custom CSV stream writer function. This is applied at the end of each transform. If not provided, a default CSV stream writer is used, and this writes to local storage. * `inputStreamOptions` Configuration options for the input stream. Supports all Papaparse CSV reader config options. * `outputStreamOptions` Configuration options for the output stream. This is only applied when using the default CSV stream writer. Supports all `toCSV` options. **Returns:** > A promise that resolves when the pipeline transformation is complete. The streamCsvTransformer can be used to [incrementally transform](https://en.wikipedia.org/wiki/Stream_processing) a CSV file. This is done by: * Streaming a CSV file from a local or **remote** path. * Passing each corresponding row as a DataFrame to the specified transformer function. * Writing the result to an output stream. [](#stream-processing-a-local-file) **Stream processing a local file** --------------------------------------------------------------------------- In the example below, we stream a local CSV file (titanic.csv), apply a transformer function, and write the output to `**titanicOutLocal.csv**`. The transformer takes each `Name` column, splits the person's title, and creates a new column from it. Node Copy import { DataFrame, Series, streamCsvTransformer } from "danfojs-node"; import path from "path" const inputFilePath = path.join(process.cwd(), "raw_data", "titanic.csv"); const outputFilePath = path.join(process.cwd(), "raw_data", "titanicOutLocal.csv"); /** * A simple function that takes a DataFrame, and transforms the Name column. * */ const transformer = (df) => { const titles = df["Name"].map((name) => name.split(".")[0]); const names = df["Name"].map((name) => name.split(".")[1]); df["Name"] = names df.addColumn("titles", titles, { inplace: true }) return df } dfd.streamCsvTransformer(inputFilePath, transformer, { outputFilePath, inputStreamOptions: { header: false } }) Output Copy //initial head of titanic.csv before transforming PassengerId,Survived,Pclass,Name,Sex,Age,SibSp,Parch,Ticket,Fare,Cabin,Embarked 1,0,3,"Braund, Mr. Owen Harris",male,22,1,0,A/5 21171,7.25,,S 2,1,1,"Cumings, Mrs. John Bradley (Florence Briggs Thayer)",female,38,1,0,PC 17599,71.2833,C85,C 3,1,3,"Heikkinen, Miss. Laina",female,26,0,0,STON/O2. 3101282,7.925,,S //Head of titanicOutLocal.csv after transforming PassengerId,Survived,Pclass,Name,Sex,Age,SibSp,Parch,Ticket,Fare,Cabin,Embarked,titles 1,0,3, Owen Harris,male,22,1,0,A/5 21171,7.25,,S,Braund, Mr 2,1,1, John Bradley (Florence Briggs Thayer),female,38,1,0,PC 17599,71.2833,C85,C,Cumings, Mrs 3,1,3, Laina,female,26,0,0,STON/O2. 3101282,7.925,,S,Heikkinen, Miss [](#stream-processing-of-remote-file) **Stream processing of remote file** ------------------------------------------------------------------------------- In the example below, we stream a remote CSV file (titanic.csv), applies a transformer function, and write the output to the `titanicOutLocal` file. The transformer takes each `Name` column, splits the person's title, and creates a new column from it. Node Copy import { DataFrame, Series, streamCsvTransformer } from "danfojs-node"; import path from "path" const inputFilePath = "https://raw.githubusercontent.com/opensource9ja/danfojs/dev/danfojs-node/tests/samples/titanic.csv" const outputFilePath = path.join(process.cwd(), "raw_data", "titanicOutRemote.csv"); /** * A simple function that takes a DataFrame, and transforms the Name column. * */ const transformer = (df) => { const titles = df["Name"].map((name) => name.split(".")[0]); const names = df["Name"].map((name) => name.split(".")[1]); df["Name"] = names df.addColumn("titles", titles, { inplace: true }) return df } dfd.streamCsvTransformer(inputFilePath, transformer, { outputFilePath, inputStreamOptions: { header: false } }) [](#stream-processing-with-a-custom-writer) **Stream processing with a custom writer** ------------------------------------------------------------------------------------------- If you need custom control of the output writer, then you can provide a pipe-able custom writer. See [https://www.freecodecamp.org/news/node-js-streams-everything-you-need-to-know-c9141306be93/](https://www.freecodecamp.org/news/node-js-streams-everything-you-need-to-know-c9141306be93/) In the example below, we add a custom writer that logs each row. You can extend this to upload each chunk to a database, or any other function you need. Node Copy const dfd = require('danfojs-node-nightly') const path = require("path") const stream = require("stream") const inputFilePath = "https://raw.githubusercontent.com/opensource9ja/danfojs/dev/danfojs-node/tests/samples/titanic.csv" const transformer = (df) => { const titles = df["Name"].map((name) => name.split(".")[0]); const names = df["Name"].map((name) => name.split(".")[1]); df["Name"] = names df.addColumn("titles", titles, { inplace: true }) return df } let count = 0 const customWriter = function () { const csvOutputStream = new stream.Writable({ objectMode: true }) csvOutputStream._write = (chunk, encoding, callback) => { //Do anything here. For example you can write to online storage DB console.log("Chunk written: " + chunk) // Eah chunk is a row DataFrame count += 1 callback() } return csvOutputStream } dfd.streamCsvTransformer( inputFilePath, transformer, { customCSVStreamWriter: customWriter, inputStreamOptions: { header: true } }) Output Copy //Showing the last log ... Chunk written: ╔════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ Survived │ Pclass │ Name │ Sex │ Age │ Siblings/Spouse… │ Parents/Childre… │ Fare │ titles ║ ╟────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 884 │ 0 │ 3 │ Patrick Dooley │ male │ 32 │ 0 │ 0 │ 7.75 │ Mr ║ ╚════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ [Previousdanfo. convertFunctionTotransformer](/api-reference/general-functions/danfo.-convertfunctiontotransformer) [Nextdanfo.streamJSON](/api-reference/general-functions/danfo.streamjson) Last updated 2 years ago Was this helpful? --- # danfo.toDateTime | Danfo.js danfo.**toDateTime**(data) Parameters Type Description Default **data** Array, Series **data**: Array Series with Date strings to convert to Date time. [](#examples) **Examples** ------------------------------- In the following example, we convert a **Series** of Date strings to DateTime objects, so we can call various Date methods on them. NodeBrowser Copy const dfd = require('danfojs-node') let data = new dateRange({ "start": '1/1/2018', period: 12, freq: 'M' }) let sf = new Series(data) sf.print() let dt = toDateTime(data) dt.dayOfMonth().print() dt.dayOfWeekName().print() dt.hours().print() Copy Output Copy ╔═══╤════════════════════════╗ ║ 0 │ 1/1/2018, 12:00:00 AM ║ ╟───┼────────────────────────╢ ║ 1 │ 2/1/2018, 12:00:00 AM ║ ╟───┼────────────────────────╢ ║ 2 │ 3/1/2018, 12:00:00 AM ║ ╟───┼────────────────────────╢ ║ 3 │ 4/1/2018, 12:00:00 AM ║ ╟───┼────────────────────────╢ ║ 4 │ 5/1/2018, 12:00:00 AM ║ ╟───┼────────────────────────╢ ║ 5 │ 6/1/2018, 12:00:00 AM ║ ╟───┼────────────────────────╢ ║ 6 │ 7/1/2018, 12:00:00 AM ║ ╟───┼────────────────────────╢ ║ 7 │ 8/1/2018, 12:00:00 AM ║ ╟───┼────────────────────────╢ ║ 8 │ 9/1/2018, 12:00:00 AM ║ ╟───┼────────────────────────╢ ║ 9 │ 10/1/2018, 12:00:00 AM ║ ╚═══╧════════════════════════╝ ╔═══╤═══╗ ║ 0 │ 1 ║ ╟───┼───╢ ║ 1 │ 1 ║ ╟───┼───╢ ║ 2 │ 1 ║ ╟───┼───╢ ║ 3 │ 1 ║ ╟───┼───╢ ║ 4 │ 1 ║ ╟───┼───╢ ║ 5 │ 1 ║ ╟───┼───╢ ║ 6 │ 1 ║ ╟───┼───╢ ║ 7 │ 1 ║ ╟───┼───╢ ║ 8 │ 1 ║ ╟───┼───╢ ║ 9 │ 1 ║ ╚═══╧═══╝ ╔═══╤═══════════╗ ║ 0 │ Monday ║ ╟───┼───────────╢ ║ 1 │ Thursday ║ ╟───┼───────────╢ ║ 2 │ Thursday ║ ╟───┼───────────╢ ║ 3 │ Sunday ║ ╟───┼───────────╢ ║ 4 │ Tuesday ║ ╟───┼───────────╢ ║ 5 │ Friday ║ ╟───┼───────────╢ ║ 6 │ Sunday ║ ╟───┼───────────╢ ║ 7 │ Wednesday ║ ╟───┼───────────╢ ║ 8 │ Saturday ║ ╟───┼───────────╢ ║ 9 │ Monday ║ ╚═══╧═══════════╝ ╔═══╤═══╗ ║ 0 │ 0 ║ ╟───┼───╢ ║ 1 │ 0 ║ ╟───┼───╢ ║ 2 │ 0 ║ ╟───┼───╢ ║ 3 │ 0 ║ ╟───┼───╢ ║ 4 │ 0 ║ ╟───┼───╢ ║ 5 │ 0 ║ ╟───┼───╢ ║ 6 │ 0 ║ ╟───┼───╢ ║ 7 │ 0 ║ ╟───┼───╢ ║ 8 │ 0 ║ ╟───┼───╢ ║ 9 │ 0 ║ ╚═══╧═══╝ Date time properties of Series or datetime-like columns in DataFrame can be accessed via accessors in the **dt** name-space. See [Accessors](https://app.gitbook.com/@jsdata/s/danfojs/~/drafts/-MEMaWwva1cjt8CxnG-b/api-reference/series#accessors) [Previousdanfo.LabelEncoder](/api-reference/general-functions/danfo.labelencoder) [Nextdanfo.getDummies](/api-reference/general-functions/danfo.get_dummies) Last updated 3 years ago Was this helpful? --- # danfo.MinMaxScaler | Danfo.js class danfo.**MinMaxScaler** danfo.js provides the MinMaxScaler class for standardization of DataFrame and Series. This estimator scales and translates each feature individually such that it is in the given range on the training set, e.g. between zero and one. This transformation is often used as an alternative to zero mean, unit variance scaling like [Standardscaler](/api-reference/general-functions/danfo.standardscaler) . The API is similar to sklearn's [MinMaxScaler](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MinMaxScaler.html?highlight=minmaxscaler#sklearn.preprocessing.MinMaxScaler) , and provides a fit and transform method. [](#examples) **Examples** ------------------------------- ### [](#standardize-dataframe-object-using-minmaxscaler) Standardize DataFrame Object using MinMaxScaler NodeBrowser Copy const dfd = require("danfojs-node") let scaler = new dfd.MinMaxScaler() let data = [[100,1000,2000, 3000] ,\ [20, 30, 20, 10],\ [1, 1, 1, 0]] let df = new dfd.DataFrame(data) df.print() scaler.fit(df) let df_enc = scaler.transform(df) df_enc.print() Copy Output Copy ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ 0 │ 1 │ 2 │ 3 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ 100 │ 1000 │ 2000 │ 3000 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ 20 │ 30 │ 20 │ 10 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ 1 │ 1 │ 1 │ 0 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ Shape: (3,4) ╔═══╤═══════════════════╤═══════════════════╤═══════════════════╤═══════════════════╗ ║ │ 0 │ 1 │ 2 │ 3 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 0 │ 1 │ 1 │ 1 │ 1 ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 1 │ 0.19191919267... │ 0.02902902849... │ 0.00950475223... │ 0.00333333341... ║ ╟───┼───────────────────┼───────────────────┼───────────────────┼───────────────────╢ ║ 2 │ 0 │ 0 │ 0 │ 0 ║ ╚═══╧═══════════════════╧═══════════════════╧═══════════════════╧═══════════════════╝ ### [](#standardize-series-object-using-minmaxscaler) Standardize Series Object Using MinMaxScaler NodeBrowser Copy const dfd = require("danfojs-node") let scaler = new dfd.MinMaxScaler() let data = [[100,1000,2000, 3000] ,\ [20, 30, 20, 10],\ [1, 1, 1, 0]] let df = new dfd.DataFrame(data) let sf = df.iloc({columns: [0]}) scaler.fit(sf) let df_enc = scaler.transform(sf) df_enc.print() Copy Output Copy Shape: (3,1) ╔═══╤═══════════════════╗ ║ │ 0 ║ ╟───┼───────────────────╢ ║ 0 │ 1 ║ ╟───┼───────────────────╢ ║ 1 │ 0.19191919267... ║ ╟───┼───────────────────╢ ║ 2 │ 0 ║ ╚═══╧═══════════════════╝ See also [MinMaxScaler](/api-reference/general-functions/danfo.minmaxscaler) [Previousdanfo.StandardScaler](/api-reference/general-functions/danfo.standardscaler) [Nextdanfo.LabelEncoder](/api-reference/general-functions/danfo.labelencoder) Last updated 3 years ago Was this helpful? ---