Crysda::DataFrame

Returns a column by name

Returns a column by index

Adds new variables and preserves existing

Add a new column and preserve existing ones.

df.add_column("salary_category") { 3 }             # with constant value
df.add_column("age_3y_later") { |e| e["age"] + 3 } # by doing basic column arithmetics

add multiple columns

df.add_columns(
  "age_plus3".with { |e| e["age"] + 3 },
  "initials".with { |e| e["first_name"].map(&.to_s[0]).concatenate(e["last_name"].map(&.to_s[0])) }
)

Returns a DataFrame containing the new row. The new row length must match the number of columns in the DataFrame

Add the row-number as column to data-frame

Apply a function to each row, returning a new DataFrame with the result column added The block receives a DataFrameRow and should return a scalar value

df.apply_rows("total") { |row| row["price"].as_f * row["qty"].as_i }
df.apply_rows("full_name") { |row| "#{row["first"].as_s} #{row["last"].as_s}" }

Add new columns. rows are matched by position, so all data frames must have the same number of rows.

Adds new rows. Missing entries are set to null. The output of bind_rows will contain a column if that column appears in any of the inputs. When row-binding, columns are matched by name, and any missing columns will be filled with NA. Grouping will be discarded when binding rows

Add new rows. Missing entries are set to nil. The output of bind_rows will contain a column if that column appears in any of the inputs. when row-binding, columns are matched by name, and any missing column will be filled with NA Grouping will be discarded when binding rows

row1 = {
  "person" => "james",
  "year"   => 1996,
  "weight" => 54.0,
  "sex"    => "M",
} of String => Any

row2 = {
  "person" => "nell",
  "year"   => 1997,
  "weight" => 48.1,
  "sex"    => "F",
} of String => Any
df.bind_rows(row1, row2)

Get first non-null value from multiple columns (coalesce)

Ordered list of column in this data-frame

Turns implicit missing values into explicit missing values. This is a wrapper around #expand

Counts observations by group.

If no grouping attributes are provided the method will respect the grouping of the receiver, or in cases of an ungrouped receiver will simply count the rows in the data.frame

selects : The variables to to be used for cross-tabulation. name : The name of the count column resulting table.

df.count("column name")

Counts expressions

If no grouping attributes are provided the method will respect the grouping of the receiver, or in cases of an ungrouped receiver will simply count the rows in the data.frame

Describe numeric columns with summary statistics

Retains only unique/distinct rows selects : Variables to use when determining uniqueness. If there are multiple rows for a given combination of inputs, only the first row will be preserved.

Drop duplicate rows

Drop rows containing any null values If columns are specified, only check those columns for nulls

Drop rows containing any null values in specified columns

Get duplicate rows

expand is often useful in conjunction with left_join if you want to convert implicit missing values to explicit missing values.

Fill null values in specific columns

Fill null values in specific columns using a hash of column => value

Fill null values in all columns with a default value Returns a new DataFrame with nulls replaced

Filter the rows of a table with a single predicate. the filter() function is used to subset a data frame, retaining all rows that satisfy your conditions.

AND-filter a table with different filters. Subset rows with filter

df.filter { |e| e.["age"] == 23 }
df.filter { |e| e.["weight"] > 50 }
df.filter { |e| e["first_name"].matching { |e| e.starts_with?("Ho") } }

filter rows by Row predicate, which is invoked on each row of the dataframe

df = Crysda.dataframe_of("person", "year", "weight", "sex").values(
  "max", 2014, 33.1, "M",
  "max", 2016, nil, "M",
  "anna", 2015, 39.2, "F",
  "anna", 2016, 39.9, "F"
)
df.filter_by_row { |f| f["year"].as_i > 2015 }.print

filter rows by Row predicate, which is invoked on each row of the dataframe

df = Crysda.dataframe_of("person", "year", "weight", "sex").values(
  "max", 2014, 33.1, "M",
  "max", 2016, nil, "M",
  "anna", 2015, 39.2, "F",
  "anna", 2016, 39.9, "F"
)
df.filter_by_row_with_index { |f, i| f["year"].as_i > 2015 || i % 2 != 0 }.print

gather takes multiple columns and collapses into key-value pairs, duplicating all other columns as needed. You use gather() when you notice that you have columns that are not variables.

key Name of the key column to create in output. value Name of the value column to create in output. columns The colums to gather. The same selectar syntax as for krangl::select is supported here convert If TRUE will automatically run convertType on the key column. This is useful if the column names are actually numeric, integer, or logical.

Creates a grouped data-frame given a list of grouping attributes. Most data operations are done on groups defined by variables. group_by() takes the receiver data-frame and converts it into a grouped data-frame where operations are performed "by group". ungroup() removes grouping.

Most verbs like add_column(), summarize(), etc. will be executed per group if a grouping is present.

Creates a grouped data-frame given a list of grouping attributes. Most data operations are done on groups defined by variables. group_by() takes the receiver data-frame and converts it into a grouped data-frame where operations are performed "by group". ungroup() removes grouping.

Most verbs like add_column(), summarize(), etc. will be executed per group if a grouping is present.

Creates a grouped data-frame from a column selector function. See select() for details about column selection.

Most data operations are done on groups defined by variables. group_by() takes the receiver data-frame and converts it into a grouped data-frame where operations are performed "by group". ungroup() removes grouping.

Creates a grouped data-frame from one or more table expressions. See add_column() for details about table expressions.

Most data operations are done on groups defined by variables. group_by() takes the receiver data-frame and converts it into a grouped data-frame where operations are performed "by group". ungroup() removes grouping.

Returns a data-frame of distinct grouping variable tuples for a grouped data-frame. An empty data-frame for ungrouped data

Returns the groups of a grouped data frame or just a reference to self

return the top rows from dataframe. default to 5

Push some columns to the left end of a data-frame

Push some columns to the right end of a data-frame

Ordered list of column names of this data-frame

Nest repeated values in a list-variable.

There are many possible ways one could choose to nest colSelect inside a data frame. nest() creates a list of data frames containing all the nested variables: this seems to be the most useful form in practice.

Usage

nest(data, ..., column_name = "data")

col_select - A selection of col_select. If not provided, all except the grouping variables are selected. column_name - The name of the new column, as a string or symbol. also see https://github.com/tidyverse/tidyr/blob/master/R/nest.R

Get the n largest rows by column value

df.nlargest(10, "sales")        # Top 10 by sales
df.nlargest(5, "score", "name") # Top 5 by score, ties broken by name

Get the n smallest rows by column value

df.nsmallest(10, "price")      # Bottom 10 by price
df.nsmallest(5, "age", "name") # Bottom 5 by age, ties broken by name

Number of columns in this dataframe

Number of rows in this dataframe

Pivot table - reshape data by aggregating values index: Column(s) to use as row index columns: Column to pivot into new columns values: Column containing values to aggregate aggfunc: Aggregation function ("sum", "mean", "count", "min", "max")

df.pivot_table("region", "product", "sales", "sum")

Prints a dataframe to output (defaults to STDOUT). df.to_s will also work but has no options

reject column by column type

reject selected columns

remove selected columns

Select or reject columns by predicate

Rename one or several columns. Positions should be preserved.

Rename one or several columns. Positions should be preserved.

Returns a row by index

Returns array of row numbers starting from 1

Returns an Iterator over all rows. Per row data is represented as DataFrameRow

Creates a grouped data-frame where each group consists of exactly one line.

Random sample of rows

Random sample by fraction

Select random rows from a table. If receiver is grouped, sampling is done per group. fraction - Fraction of rows to sample replace - Sample with or without replacement

Select random rows from a table. If receiver is grouped, sampling is done per group. n - Number of rows to sample replace - Sample with or without replacement

Prints the schema (that is column names, types, and the first few values per column) of a dataframe to output (defaults to STDOUT).

Create a new data frame with only selected columns

Select column by column type

Keeps only the variables that match any of the given expression

Select or reject columns by predicate

Special case of inner join against distinct right side

Given either regular expression or a vector of character positions, separate() turns a single character column into multiple columns.

column - Bare column name. into - Names of new variables to create as character vector. sep - Separator between columns. If String, is interpreted as a regular expression. The default value is a regular expression that matches any sequence of non-alphanumeric values. remove - If true, remove input column from output data frame. convert - If set, attempt to do a type conversion will be run on all new columns. This is useful if the value column was a mix of variables that was coerced to a string.

Replace current column names with new ones. The number of provided names must match the number of columns.

Replace current column names with new ones. The number of provided names must match the number of columns.

Shuffle rows randomly

Select rows by position while taking into account grouping in a data-frame.

Select rows by position while taking into account grouping in a data-frame.

Resorts the receiver in ascending order (small values to go top of table). The first argument defines the primary attribute to sort by. Additional ones are used to resolve ties.

Missing values will come last in the sorted table.

Resorts the receiver in descending order (small values to go bottom of table). The first argument defines the primary attribute to sort by. Additional ones are used to resolve ties.

spread a key-value pair across multiple columns.

key The bare (unquoted) name of the column whose values will be used as column headings. value The bare (unquoted) name of the column whose values will populate the cells. fill If set, missing values will be replaced with this value - NOT IMPLEMENTED convert If set, attempt to do a type conversion will be run on all new columns. This is useful if the value column was a mix of variables that was coerced to a string.

Creates a summary of a table or a group. The provided expression is expected to evaluate to a scalar value and not into a column. summarize() is typically used on grouped data created by group_by(). The output will have one row for each group.

Expose a view on the data as Hash from column names to nullable arrays.

Convert DataFrame to JSON string

Converts dataframe to its string representation. This is being invoked via print and to_s

Create a new dataframe based on a list of column-formulas which are evaluated in the context of the this instance.

Removes the grouping (if present from a data frame)

Convenience function to paste together multiple columns into one.

colName - Name of the column to add which - Names of columns which should be concatenated together sep - Separator to use between values. remove - If true, remove input columns from output data frame.

see separate

If you have a list-column, this makes each element of the list its own row. It unfolds data vertically. unnest() can handle list-columns that can atomic vectors, lists, or data frames (but not a mixture of the different types).

Value counts for a column - returns DataFrame with value and count

Save the current dataframe to separator delimited file.

Write DataFrame to JSON file (array of objects format)

df.write_json("output.json")
df.write_json("output.json", pretty: true)

Write DataFrame to IO as JSON (array of objects format)