Class: BatchFileLoad::Import

Inherits:
Object
  • Object
show all
Defined in:
lib/batch_file_load/import.rb

Direct Known Subclasses

Import::Otus::SimpleInterpreter

Instance Attribute Summary (collapse)

Instance Method Summary (collapse)

Constructor Details

- (Import) initialize(project_id: nil, user_id: nil, files: nil, import_level: :warn)

Returns a new instance of Import

Parameters:

  • project_id (Integer)
  • user_id (Integer)
  • files (Array)
  • import_level (Symbol)


14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
# File 'lib/batch_file_load/import.rb', line 14

def initialize(project_id: nil, user_id: nil, files: nil, import_level: :warn)
  @project_id = project_id
  @user = User.find(user_id)
  @files = files
  @import_level = import_level

  @processed = false

  # WARNING: Beware of files with the same name, the content within them may be different
  # thus why filenames can NOT be used as keys for this reason
  # We also can't modify filenames by appending numbers at the end or whatever to
  # fix the previous issue in case the filename contains metadata for the file
  # in which this would break the metadata
  @processed_files = { names: [], objects: [] }

  @filenames = []
  @file_contents = []

  @files.each do |file|
    @filenames.push(file.original_filename)

    # WARNING: Once you call ".tempfile.read.force_encoding('utf-8')" on a tempfile as shown below,
    # the next time you call ".tempfile.read.force_encoding('utf-8')" on the same tempfile
    # an empty string will be returned!
    @file_contents.push(file.tempfile.read.force_encoding('utf-8'))
  end

  @errors = []
  @file_errors = []

  @total_records_created = 0

  build
end

Instance Attribute Details

- (Object) errors (readonly)

Returns the value of attribute errors



6
7
8
# File 'lib/batch_file_load/import.rb', line 6

def errors
  @errors
end

- (Object) file_contents (readonly)

Returns the value of attribute file_contents



9
10
11
# File 'lib/batch_file_load/import.rb', line 9

def file_contents
  @file_contents
end

- (Object) file_errors (readonly)

Returns the value of attribute file_errors



7
8
9
# File 'lib/batch_file_load/import.rb', line 7

def file_errors
  @file_errors
end

- (Object) filenames (readonly)

Returns the value of attribute filenames



8
9
10
# File 'lib/batch_file_load/import.rb', line 8

def filenames
  @filenames
end

- (Object) processed (readonly)

Returns the value of attribute processed



5
6
7
# File 'lib/batch_file_load/import.rb', line 5

def processed
  @processed
end

- (Object) processed_files (readonly)

Returns the value of attribute processed_files



4
5
6
# File 'lib/batch_file_load/import.rb', line 4

def processed_files
  @processed_files
end

Instance Method Details

- (Object) build (protected)

Subclass implemented function that is responsible for interpreting the imported data from the files



124
125
126
# File 'lib/batch_file_load/import.rb', line 124

def build
  raise 'This method must be provided in the respective subclass.'
end

- (Object) create

Attempts to save each object from the files into the database



50
51
52
53
54
55
56
57
58
59
60
61
62
63
# File 'lib/batch_file_load/import.rb', line 50

def create
  if ready_to_create?
    @total_records_created = 0

    get_all_objects.each do |object|
      if object.save
        @total_records_created += 1
      end
    end
  else
    @errors << "Import level #{@import_level} has prevented creation." unless import_level_ok?
    @errors << 'One of project_id, user_id or files has not been provided.' unless valid?
  end
end

- (Boolean) file_object_strict_level_ok?

There must be records from every file and every record must be valid

Returns:

  • (Boolean)


117
118
119
# File 'lib/batch_file_load/import.rb', line 117

def file_object_strict_level_ok?
  file_strict_level_ok? && object_strict_level_ok?
end

- (Boolean) file_strict_level_ok?

There must be records from each file

Returns:

  • (Boolean)


97
98
99
100
101
# File 'lib/batch_file_load/import.rb', line 97

def file_strict_level_ok?
  @filenames.each_with_index do |filename, index|
    return false if filename != @processed_files[:names][index] || @processed_files[:objects][index].empty?
  end
end

- (Array) get_all_objects (private)

Returns an array that has every object from each file

Returns:

  • (Array)


155
156
157
158
159
160
161
162
163
164
165
166
167
# File 'lib/batch_file_load/import.rb', line 155

def get_all_objects
  all_objects = []

  @processed_files[:objects].each do |hash|
    hash.each_value do |objects|
      objects.each do |object|
        all_objects.push(object)
      end
    end
  end

  all_objects
end

- (Boolean) import_level_ok? (private)

Checks if a file passes the specificed import level

Returns:

  • (Boolean)


138
139
140
141
142
143
144
145
146
147
148
149
150
151
# File 'lib/batch_file_load/import.rb', line 138

def import_level_ok?
  case @import_level.to_sym
  when :warn
    warn_level_ok?
  when :file_strict
    file_strict_level_ok?
  when :object_strict
    object_strict_level_ok?
  when :file_object_strict
    file_object_strict_level_ok?
  else
    false
  end
end

- (Boolean) object_strict_level_ok?

Every record must be valid

Returns:

  • (Boolean)


105
106
107
108
109
110
111
112
113
# File 'lib/batch_file_load/import.rb', line 105

def object_strict_level_ok?
  begin
    get_all_objects.each do |object|
      return false if !object.valid?
    end
  rescue ArgumentError 
    return false
  end
end

- (Boolean) ready_to_create? (private)

Returns true if ready to create all the objects and store in the database

Returns:

  • (Boolean)


132
133
134
# File 'lib/batch_file_load/import.rb', line 132

def ready_to_create?
  valid? && @processed && import_level_ok?
end

- (Integer) total_files_processed

Returns the number of files that got processed

Returns:

  • (Integer)


79
80
81
# File 'lib/batch_file_load/import.rb', line 79

def total_files_processed
  @processed_files[:names].length
end

- (Integer) total_records_created

Returns the number of objects that were successfully saved to the database

Returns:

  • (Integer)


67
68
69
# File 'lib/batch_file_load/import.rb', line 67

def total_records_created
  @total_records_created
end

- (Integer) total_records_processed

Returns the total number of projects that got processed from each file

Returns:

  • (Integer)


73
74
75
# File 'lib/batch_file_load/import.rb', line 73

def total_records_processed
  get_all_objects.length
end

- (Boolean) valid?

Checks if valid housekeeping and file attributes were supplied

Returns:

  • (Boolean)


85
86
87
# File 'lib/batch_file_load/import.rb', line 85

def valid?
  @project_id && @user && @filenames && @file_contents
end

- (Boolean) warn_level_ok?

Anything can happen

Returns:

  • (Boolean)


91
92
93
# File 'lib/batch_file_load/import.rb', line 91

def warn_level_ok?
  true
end