Module: Download

Defined in:
lib/download.rb

Overview

Code that translates scopes into downloadable tab-delimited CSV. Dependant on Postgresql.

Class Method Summary (collapse)

Class Method Details

+ (CSV::Table) delete_columns(tbl, columns = [])

Returns delete the specified columns

Parameters:

  • table (CSV::Table)
  • columns (Array) (defaults to: [])

Returns:

  • (CSV::Table)

    delete the specified columns



45
46
47
48
49
50
51
# File 'lib/download.rb', line 45

def self.delete_columns(tbl, columns = [])
  return tbl if columns.empty?
  columns.each do |col|
    tbl.delete(col.to_s)
  end
 tbl
end

+ (CSV) generate_csv(scope, exclude_columns: [], header_converters: [], trim_rows: false, trim_columns: false)

translate a scope into a CSV table, with optional tweaks to the data

This is a very nice reference for future consideration:

http://collectiveidea.com/blog/archives/2015/03/05/optimizing-rails-for-memory-usage-part-3-pluck-and-database-laziness

Parameters:

  • scope (Scope)
  • exclude_columns (Array)
  • header_converters (Array)
  • trim_rows (Boolean)
  • trim_columns (Boolean)

Returns:

  • (CSV)


15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
# File 'lib/download.rb', line 15

def self.generate_csv(scope, exclude_columns: [], header_converters: [], trim_rows: false, trim_columns: false)

  # Check to see if keys is deterministicly ordered
  column_names = scope.columns_hash.keys

  h = CSV.new(column_names.join(','), header_converters: header_converters, headers: true)
  h.read

  headers = CSV::Row.new(h.headers, h.headers, true)

  tbl = CSV::Table.new([headers])

  # Pluck rows is from postgresql_cursor gem
  scope.pluck_rows(*column_names).each do |o|
    tbl << o.collect{|v| Utilities::Strings.sanitize_for_csv(v) }
    # If keys are not deterministic: .attributes.values_at(*column_names).collect{|v| Utilities::Strings.sanitize_for_csv(v) }
  end

  Download.delete_columns(tbl, exclude_columns) if !exclude_columns.empty?
  Download.trim_columns(tbl) if trim_columns
  Download.trim_rows(tbl) if trim_rows

  # CSV::Converters are only available on read, not generate, so we can't use them here.
  tbl.to_csv(col_sep: "\t", encoding: Encoding::UTF_8)
end

+ (CSV::Table) trim_columns(table)

Parameters:

  • table (CSV::Table)

    remove columns without any non-#blank? values (of course doing this in the scope is better!)

Returns:

  • (CSV::Table)


75
76
77
78
79
80
81
82
83
84
85
86
87
# File 'lib/download.rb', line 75

def self.trim_columns(table)
  table.by_col!.delete_if do |h, col|
    d = true
    col.shift
    col.each do |v|
      next if v.blank?
      d = false
      break
    end
    d
  end
  table
end

+ (CSV::Table) trim_rows(table, skip_id = true)

Parameters:

  • table (CSV::Table)
  • skip_id (Boolean) (defaults to: true)

    if true ignore the 'id'l column in consideration of whether to delete this row delete rows if there are no values in any cell (of course doing this in the scope is better!)

Returns:

  • (CSV::Table)


57
58
59
60
61
62
63
64
65
66
67
68
69
70
# File 'lib/download.rb', line 57

def self.trim_rows(table, skip_id = true)
  headers = table.headers
  headers = headers - ['id'] if skip_id
  table.by_row!.delete_if do |row|
    d = true
    headers.each do |h|
      next if row[h].blank?
      d = false
      break
    end
    d
  end
  table
end