Class: DwcOccurrence
- Inherits:
-
ApplicationRecord
- Object
- ActiveRecord::Base
- ApplicationRecord
- DwcOccurrence
- Includes:
- Housekeeping
- Defined in:
- app/models/dwc_occurrence.rb
Overview
A Darwin Core Record for the Occurrence core. Field generated from Ruby dwc-meta, which references the same spec that is used in the IPT, and the Dwc Assistant. Each record references a specific CollectionObject or AssertedDistribution.
Important: This is a cache/index, data here are periodically destroyed and regenerated from multiple tables in TW.
DWC attributes are camelCase to facilitate matching dwcClass is a replacement for the Rails reserved 'Class'
All DC attributes (attributes not in DwcOccurrence::TW_ATTRIBUTES) in this table are namespaced to dc (“purl.org/dc/terms/”, “rs.tdwg.org/dwc/terms/”)
README:
There is a two part strategy to building the index. 1) An individual record will rebuild on request with `parameter to collection_objects/123/dwc*?build=true`.
2) Wipe, and rebuild on some schedule. It would in theory be possible to track and rebuild when a class of every property was created (or updated), however
this is a lot of overhead to inject/code for a lot of models. It would inject latency at numerous stages that would perhaps impact UI performance.
Notes -
This experiment isn't going to work at scale, too many preceding calls for big returns
after_find :refresh, if: -> { persisted? && is_stale? }
def refresh
dwc_occurrence_object.set_dwc_occurrence
end
TODO: The basisOfRecord CVTs are not super informative.
We know collection object is definitely 1:1 with PreservedSpecimen, however
AssertedDistribution could be HumanObservation (if source is person), or ... what? if
its a published record. Seems we need a 'PublishedAssertation', just like we model the data.
Gotchas.
* updated_at is set by touching the record, not via housekeeping.
Constant Summary collapse
- DC_NAMESPACE =
'http://rs.tdwg.org/dwc/terms/'.freeze
- TW_ATTRIBUTES =
Not yet implemented, but likely needed (at an even higher level) ? :id
[ :id, :project_id, :created_at, :updated_at, :created_by_id, :updated_by_id, :dwc_occurrence_object_type, :dwc_occurence_object_id ].freeze
- HEADER_CONVERTERS =
{ 'dwcClass' => 'class', }.freeze
Instance Attribute Summary collapse
-
#occurrence_identifier ⇒ Object
Returns the value of attribute occurrence_identifier.
Class Method Summary collapse
- .annotates? ⇒ Boolean
-
.asserted_distributions_join ⇒ ActiveRecord::Relation
that matches, consider moving to Shared.
-
.by_collection_object_filter(filter_scope: nil, project_id: nil) ⇒ Object
Return scopes by a collection object filter.
-
.collection_objects_join ⇒ ActiveRecord::Relation
that matches, consider moving to Shared.
-
.computed_columns ⇒ Scope
The columns inferred to have data.
-
.empty_fields ⇒ Array
Of column names as symbols that are blank in ALL projects (not just this one).
-
.excluded_columns ⇒ Array
Of symbols.
-
.scoped_by_otu(otu) ⇒ Scope
All DwcOccurrences for the Otu * Includes synonymy (coordinate OTUs).
-
.target_columns ⇒ Array
!! TODO: When we come to adding AssertedDistributions, FieldOccurrnces, etc.
Instance Method Summary collapse
- #basis ⇒ Object
- #create_object_uuid ⇒ Object protected
-
#generate_uuid_if_required(force = false) ⇒ Object
TODO: quick check if occurrenceID exists in table?! <-> locking sync !?.
-
#is_stale? ⇒ Boolean
!! This a spot check, it's not (yet) coded to be comprehensive.
- #set_metadata_attributes ⇒ Object protected
-
#taxon_determination_is_stale? ⇒ Boolean
Boolean acts_as_list actions don't trigger updated_at changes, handle this by comparing current values.
- #uuid_identifier_scope ⇒ Object
Methods included from Housekeeping
#has_polymorphic_relationship?
Methods inherited from ApplicationRecord
Instance Attribute Details
#occurrence_identifier ⇒ Object
Returns the value of attribute occurrence_identifier.
75 76 77 |
# File 'app/models/dwc_occurrence.rb', line 75 def occurrence_identifier @occurrence_identifier end |
Class Method Details
.annotates? ⇒ Boolean
77 78 79 |
# File 'app/models/dwc_occurrence.rb', line 77 def self.annotates? false end |
.asserted_distributions_join ⇒ ActiveRecord::Relation
that matches, consider moving to Shared
95 96 97 98 99 100 |
# File 'app/models/dwc_occurrence.rb', line 95 def self.asserted_distributions_join a = arel_table b = ::AssertedDistribution.arel_table j = a.join(b).on(a[:dwc_occurrence_object_type].eq('AssertedDistribution').and(a[:dwc_occurrence_object_id].eq(b[:id]))) joins(j.join_sources) end |
.by_collection_object_filter(filter_scope: nil, project_id: nil) ⇒ Object
Return scopes by a collection object filter
138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 |
# File 'app/models/dwc_occurrence.rb', line 138 def self.by_collection_object_filter(filter_scope: nil, project_id: nil) return DwcOccurrence.none if project_id.nil? || filter_scope.nil? c = ::CollectionObject.arel_table d = arel_table # TODO: hackish k = ::CollectionObject.select('coscope.id').from( '(' + filter_scope.to_sql + ') as coscope ' ) a = self.collection_objects_join .where('dwc_occurrences.project_id = ?', project_id) .where(dwc_occurrence_object_id: k) .select(::DwcOccurrence.target_columns) # TODO !! Will have to change when AssertedDistribution and other types merge in a end |
.collection_objects_join ⇒ ActiveRecord::Relation
that matches, consider moving to Shared
86 87 88 89 90 91 |
# File 'app/models/dwc_occurrence.rb', line 86 def self.collection_objects_join a = arel_table b = ::CollectionObject.arel_table j = a.join(b).on(a[:dwc_occurrence_object_type].eq('CollectionObject').and(a[:dwc_occurrence_object_id].eq(b[:id]))) joins(j.join_sources) end |
.computed_columns ⇒ Scope
Returns the columns inferred to have data.
190 191 192 |
# File 'app/models/dwc_occurrence.rb', line 190 def self.computed_columns select(target_columns) end |
.empty_fields ⇒ Array
Returns of column names as symbols that are blank in ALL projects (not just this one).
156 157 158 159 160 161 162 163 164 165 166 167 |
# File 'app/models/dwc_occurrence.rb', line 156 def self.empty_fields empty_in_all_projects = ActiveRecord::Base.connection.execute("select attname from pg_stats where tablename = 'dwc_occurrences' and most_common_vals is null and most_common_freqs is null and histogram_bounds is null and correlation is null and null_frac = 1;").pluck('attname').map(&:to_sym) empty_in_all_projects # - target_columns end |
.excluded_columns ⇒ Array
Returns of symbols.
184 185 186 |
# File 'app/models/dwc_occurrence.rb', line 184 def self.excluded_columns ::DwcOccurrence.columns.collect{|c| c.name.to_sym} - (self.target_columns - [:dwc_occurrence_object_id, :dwc_occurrence_object_type]) end |
.scoped_by_otu(otu) ⇒ Scope
Returns all DwcOccurrences for the Otu
-
Includes synonymy (coordinate OTUs).
107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 |
# File 'app/models/dwc_occurrence.rb', line 107 def self.scoped_by_otu(otu) a,b = nil, nil if otu.taxon_name_id.present? a = ::Queries::DwcOccurrence::Filter.new( asserted_distribution_query: { taxon_name_query: { taxon_name_id: otu.taxon_name_id, descendants: false, synonymify: true } }) b = ::Queries::DwcOccurrence::Filter.new( collection_object_query: { taxon_name_query: { taxon_name_id: otu.taxon_name_id, descendants: false, synonymify: true } }) else a = ::Queries::DwcOccurrence::Filter.new( asserted_distribution_query: { otu_id: otu.id}) b = ::Queries::DwcOccurrence::Filter.new( collection_object_query: { otu_query: { otu_id: otu.id}}) end from("((#{a.all.to_sql}) UNION (#{b.all.to_sql})) as dwc_occurrences") end |
.target_columns ⇒ Array
!! TODO: When we come to adding AssertedDistributions, FieldOccurrnces, etc. we will have to make this more flexible
173 174 175 176 177 178 179 180 |
# File 'app/models/dwc_occurrence.rb', line 173 def self.target_columns [:id, # must be in position 0 :occurrenceID, :basisOfRecord, :dwc_occurrence_object_id, # !! We don't want this, but need it in joins, it is removed in trim via `.excluded_columns` below :dwc_occurrence_object_type, # !! ^ ] + CollectionObject::DwcExtensions::DWC_OCCURRENCE_MAP.keys end |
Instance Method Details
#basis ⇒ Object
194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 |
# File 'app/models/dwc_occurrence.rb', line 194 def basis case dwc_occurrence_object_type when 'CollectionObject' if dwc_occurrence_object.is_fossil? return 'FossilSpecimen' else return 'PreservedSpecimen' end when 'AssertedDistribution' # Used to fork b/b Source::Human and Source::Bibtex: case dwc_occurrence_object.source&.type || dwc_occurrence_object.sources.order(cached_nomenclature_date: :DESC).first.type when 'Source::Bibtex' return 'MaterialCitation' when 'Source::Human' return 'HumanObservation' else # Not recommended at this point return 'Occurrence' end end 'Undefined' end |
#create_object_uuid ⇒ Object (protected)
286 287 288 289 290 291 292 |
# File 'app/models/dwc_occurrence.rb', line 286 def create_object_uuid @occurrence_identifier = Identifier::Global::Uuid::TaxonworksDwcOccurrence.create!( identifier_object: dwc_occurrence_object, by: dwc_occurrence_object&.creator, # revisit, why required? project_id: dwc_occurrence_object&.project_id, # Current.project_id, # revisit, why required? is_generated: true) end |
#generate_uuid_if_required(force = false) ⇒ Object
TODO: quick check if occurrenceID exists in table?! <-> locking sync !?
228 229 230 231 232 233 234 235 236 |
# File 'app/models/dwc_occurrence.rb', line 228 def generate_uuid_if_required(force = false) if force # really make sure there is an object to work with create_object_uuid if !occurrence_identifier && !dwc_occurrence_object.nil? # TODO: can be simplified when inverse_of/validation added to identifiers else # assume if occurrenceID is not blank identifier is present if occurrenceID.blank? create_object_uuid if !occurrence_identifier && !dwc_occurrence_object.nil? # TODO: can be simplified when inverse_of/validation added to identifiers end end end |
#is_stale? ⇒ Boolean
!! This a spot check, it's not (yet) coded to be comprehensive. !! You should request a full rebuild (rebuild=true) at display time !! to ensure an up-to-date individual record
246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 |
# File 'app/models/dwc_occurrence.rb', line 246 def is_stale? case dwc_occurrence_object_type when 'CollectionObject' # Checks made on values, not time (necessary to handle lists) return true if taxon_determination_is_stale? # Order these roughgly by most probable/frequent changes, and/or fewest db calls required times = [ dwc_occurrence_object.updated_at, # Shouldn't be neccessary since on_save rebuilds, but cheap here dwc_occurrence_object.collecting_event&.updated_at, dwc_occurrence_object.taxon_determinations.order(:position)&.first&.updated_at, dwc_occurrence_object.biocuration_classifications.order(:updated_at).first&.updated_at, dwc_occurrence_object.georeferences.order(:updated_at).first&.updated_at, dwc_occurrence_object.data_attributes.order(:updated_at).first&.updated_at, dwc_occurrence_object.collecting_event&.data_attributes&.order(:updated_at)&.first&.updated_at, ] n = read_attribute(:updated_at) # || Time.current <- if this then never stale! times.each do |v| next if v.nil? return true if v > n end return false else # AssertedDistribution dwc_occurrence_object.updated_at > updated_at end end |
#set_metadata_attributes ⇒ Object (protected)
294 295 296 297 |
# File 'app/models/dwc_occurrence.rb', line 294 def write_attribute( :basisOfRecord, basis) write_attribute( :occurrenceID, occurrence_identifier&.identifier) end |
#taxon_determination_is_stale? ⇒ Boolean
Returns Boolean acts_as_list actions don't trigger updated_at changes, handle this by comparing current values.
280 281 282 |
# File 'app/models/dwc_occurrence.rb', line 280 def taxon_determination_is_stale? dwc_occurrence_object&.taxon_determinations.order(:position).first&.otu&.taxon_name&. != scientificName end |
#uuid_identifier_scope ⇒ Object
216 217 218 |
# File 'app/models/dwc_occurrence.rb', line 216 def uuid_identifier_scope dwc_occurrence_object&.identifiers&.where('identifiers.type like ?', 'Identifier::Global::Uuid%')&.order(:position) end |