Class: CollectionObject

Inherits:

ApplicationRecord

• Object
• ActiveRecord::Base
• ApplicationRecord
• CollectionObject

Includes:

DwcExtensions, GlobalID::Identification, Housekeeping, Shared::BiologicalExtensions, Shared::Citations, Shared::Confidences, Shared::Containable, Shared::DataAttributes, Shared::Depictions, Shared::HasPapertrail, Shared::Identifiers, Shared::IsData, Shared::IsDwcOccurrence, Shared::Loanable, Shared::Notes, Shared::Observations, Shared::OriginRelationship, Shared::ProtocolRelationships, Shared::QueryBatchUpdate, Shared::Tags, Shared::Taxonomy, SoftValidation

Defined in:

app/models/collection_object.rb

Overview

A CollectionObject is on or more physical things that have been collected. Enumerating how many things (@!total) is a task of the curator.

A CollectiongObjects immediate disposition is handled through its relation to containers. Containers can be nested, labeled, and interally subdivided as necessary.

Direct Known Subclasses

BiologicalCollectionObject

Defined Under Namespace

Modules: DwcExtensions Classes: BiologicalCollectionObject

Constant Summary

CO_OTU_HEADERS =

TODO: move to export

%w{OTU OTU\ name Family Genus Species Country State County Locality Latitude Longitude}.freeze

BUFFERED_ATTRIBUTES =

%i{buffered_collecting_event buffered_determinations buffered_other_labels}.freeze

GRAPH_ENTRY_POINTS =

[:biological_associations, :data_attributes, :taxon_determinations, :biocuration_classifications, :collecting_event, :origin_relationships, :extracts, :observation_matrices]

Constants included from Shared::IsDwcOccurrence

Shared::IsDwcOccurrence::DWC_DELIMITER, Shared::IsDwcOccurrence::VIEW_EXCLUSIONS

Constants included from SoftValidation

SoftValidation::ANCESTORS_WITH_SOFT_VALIDATIONS

Instance Attribute Summary

• #accessioned_at ⇒ Date

  The date when the object was accessioned to the Repository (not necessarily it’s current disposition!).

• #buffered_collecting_event ⇒ String

  An incoming, typically verbatim, block of data typically as seens as a locality/method/etc.

• #buffered_determinations ⇒ String

  An incoming, typically verbatim, block of data typically as seen a taxonomic determination label.

• #buffered_other_labels ⇒ String

  An incoming, typically verbatim, block of data, as typically found on label that is unrelated to determinations or collecting events.

• #collecting_event_id ⇒ Integer

  The id of the collecting event from whence this object came.

• #current_respository_id ⇒ Integer

  The id of the current repository.

• #deaccession_reason ⇒ String

  A free text explanation of why the object was removed from tracking.

• #deaccessioned_at ⇒ Date

  The date when the object was removed from tracking.

• #preparation_type_id ⇒ Integer

  How the collection object was prepared.

• #project_id ⇒ Integer

  the project ID.

• #ranged_lot_category_id ⇒ Integer

  The id of the user-defined ranged lot category.

• #respository_id ⇒ Integer

  The id of the Repository.

• #total ⇒ Integer

  The enumerated number of things, as asserted by the person managing the record.

• #type ⇒ String

  The subclass of collection object, e.g.

Class Method Summary

• .batch_update(params) ⇒ Object
• .batch_update_dwc_occurrence(params) ⇒ Object
• .bc_attributes(collection_object, col_defs) ⇒ Array

  Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object.

• .bc_headers(project_id) ⇒ Hash

  decode which headers to be displayed for biocuration classifications.

• .breakdown_buffered(collection_objects) ⇒ Hash

  A unque list of buffered_ values observed in the collection objects passed.

• .breakdown_status(collection_objects) ⇒ Object

  TODO: move to a helper.

• .ce_attributes(collection_object, col_defs) ⇒ Array

  Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object.

• .ce_headers(project_id) ⇒ Hash

  decode which headers to be displayed for collecting events.

• .co_attributes(collection_object, col_defs) ⇒ Array

  Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object.

• .co_headers(project_id) ⇒ Hash

  decode which headers to be displayed for collection objects.

• .earliest_date(project_id) ⇒ Object

  TODO: this should be refactored to be collection object centric AFTER it is spec’d.

• .from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id) ⇒ Scope

  Of intersection of collecting events (usually by date range) and collection objects (usually by inclusion in geographic areas/items).

• .in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on') ⇒ Scope

  TODO: move to filter.

• .in_geographic_item(geographic_item, limit, steps = false) ⇒ Scope

  TODO: Clarify this.

• .latest_date(project_id) ⇒ Object

  TODO: this should be refactored to be collection object centric AFTER it is spec’d.

• .select_optimized(user_id, project_id, target = nil) ⇒ Hash

  Otus optimized for user selection.

• .selected_column_names ⇒ Object

  TODO: deprecate.

• .sequence_join_hack_sql ⇒ Object

  This is a hack, maybe related to a Rails 5.1 bug.

• .used_recently(user_id, project_id, used_on = '') ⇒ Scope

  The max 10 most recently used collection_objects, as ‘used_on`.

Instance Method Summary

• #annotations ⇒ Object
• #assign_type_if_total_or_ranged_lot_category_id_provided ⇒ Object protected
• #check_that_both_of_category_and_total_are_not_present ⇒ Object protected
• #check_that_either_total_or_ranged_lot_category_id_is_present ⇒ Object protected
• #collecting_event_belongs_to_project ⇒ Object protected
• #dwc_occurrence_update_query ⇒ Object
• #geographic_name_classification ⇒ Object
• #is_biological? ⇒ Boolean

  return [Boolean] True if instance is a subclass of BiologicalCollectionObject.

• #is_image_stub? ⇒ Boolean

  See Depiction#destroy_image_stub_collection_object Used to determin if the CO can be destroy after moving an image off this object.

• #preferred_catalog_number ⇒ Identifier::Local::CatalogNumber^?

  TODO: Unify with Extract in concern.

• #preferred_record_number ⇒ Identifier::Local::RecordNumber^?

  The first (position) record_Number, on a specimen !1 Doesn’t presently support containers.

• #reject_collecting_event(attributed) ⇒ Object protected
• #sv_missing_accession_fields ⇒ Object
• #sv_missing_biocuration_classification ⇒ Object
• #sv_missing_collecting_event ⇒ Object
• #sv_missing_deaccession_fields ⇒ Object
• #sv_missing_determination ⇒ Object
• #sv_missing_preparation_type ⇒ Object
• #sv_missing_repository ⇒ Object

Methods included from DwcExtensions

#api_image_link, #dwc_associated_media, #dwc_associated_taxa, #dwc_caste, #dwc_catalog_number, #dwc_class, #dwc_collection_code, #dwc_coordinate_uncertainty_in_meters, #dwc_country, #dwc_county, #dwc_date_identified, #dwc_day, #dwc_decimal_latitude, #dwc_decimal_longitude, #dwc_end_day_of_year, #dwc_event_date, #dwc_event_id, #dwc_event_remarks, #dwc_event_time, #dwc_family, #dwc_field_number, #dwc_footprint_wkt, #dwc_genus, #dwc_geodetic_datum, #dwc_georeference_protocol, #dwc_georeference_remarks, #dwc_georeference_sources, #dwc_georeferenced_by, #dwc_georeferenced_date, #dwc_higher_classification, #dwc_identification_remarks, #dwc_identified_by, #dwc_identified_by_id, #dwc_individual_count, #dwc_infraspecific_epithet, #dwc_institution_code, #dwc_institution_id, #dwc_internal_attribute_for, #dwc_kingdom, #dwc_life_stage, #dwc_locality, #dwc_maximum_depth_in_meters, #dwc_maximum_elevation_in_meters, #dwc_minimum_depth_in_meters, #dwc_minimum_elevation_in_meters, #dwc_month, #dwc_nomenclatural_code, #dwc_occurrence_remarks, #dwc_occurrence_status, #dwc_order, #dwc_other_catalog_numbers, #dwc_phylum, #dwc_preparations, #dwc_previous_identifications, #dwc_record_number, #dwc_recorded_by, #dwc_recorded_by_id, #dwc_sampling_protocol, #dwc_scientific_name, #dwc_sex, #dwc_specific_epithet, #dwc_start_day_of_year, #dwc_state_province, #dwc_subfamily, #dwc_subtribe, #dwc_superfamily, #dwc_taxon_name_authorship, #dwc_taxon_rank, #dwc_tribe, #dwc_type_status, #dwc_verbatim_coordinates, #dwc_verbatim_depth, #dwc_verbatim_elevation, #dwc_verbatim_event_date, #dwc_verbatim_habitat, #dwc_verbatim_label, #dwc_verbatim_latitude, #dwc_verbatim_locality, #dwc_verbatim_longitude, #dwc_verbatim_srs, #dwc_water_body, #dwc_year, #is_fossil?, #set_georeference_attributes

Methods included from Shared::IsDwcOccurrence

#dwc_occurrence_attribute_values, #dwc_occurrence_attributes, #dwc_occurrence_id, #get_dwc_occurrence, #set_dwc_occurrence

Methods included from Shared::BiologicalExtensions

#missing_determination, #name_at_rank_string, #reject_otus, #reject_taxon_determinations, #requires_taxon_determination?

Methods included from SoftValidation

#clear_soft_validations, #fix_for, #fix_soft_validations, #soft_fixed?, #soft_valid?, #soft_validate, #soft_validated?, #soft_validations, #soft_validators

Methods included from Shared::QueryBatchUpdate

#query_update

Methods included from Shared::IsData

#errors_excepting, #full_error_messages_excepting, #identical, #is_community?, #is_destroyable?, #is_editable?, #is_in_use?, #is_in_users_projects?, #metamorphosize, #similar

Methods included from Shared::HasPapertrail

#attribute_updated, #attribute_updater

Methods included from Shared::ProtocolRelationships

#protocolled?, #reject_protocols

Methods included from Shared::Confidences

#reject_confidences

Methods included from Shared::OriginRelationship

#new_objects, #old_objects, #reject_origin_relationships, #set_origin

Methods included from Shared::Depictions

#has_depictions?, #image_array=, #reject_depictions, #reject_images

Methods included from Shared::Tags

#reject_tags, #tag_with, #tagged?, #tagged_with?

Methods included from Shared::Notes

#concatenated_notes_string, #reject_notes

Methods included from Shared::Identifiers

#dwc_occurrence_id, #identified?, #next_by_identifier, #previous_by_identifier, #reject_identifiers, #uri, #uuid

Methods included from Shared::Loanable

#all_loan_items, #all_loans, #container_loan_items, #container_loaned?, #container_loans, #container_times_loaned, #current_loan, #current_loan_item, #has_been_loaned?, #is_loanable?, #loan_return_date, #loaned_in_container, #on_loan?, #times_loaned

Methods included from Shared::DataAttributes

#import_attributes, #internal_attributes, #keyword_value_hash, #reject_data_attributes

Methods included from Shared::Containable

#contain, #containable?, #contained?, #contained_by?, #contained_siblings, #enclosing_containers, #put_in_container

Methods included from Shared::Citations

#cited?, #mark_citations_for_destruction, #nomenclature_date, #origin_citation_source_id, #reject_citations, #requires_citation?, #sources_by_topic_id

Methods included from Housekeeping

#has_polymorphic_relationship?

Methods inherited from ApplicationRecord

transaction_with_retry

Instance Attribute Details

#accessioned_at ⇒ Date

The date when the object was accessioned to the Repository (not necessarily it’s current disposition!). If present Repository must be present.

Returns:

• (Date)

# File 'app/models/collection_object.rb', line 64

class CollectionObject < ApplicationRecord
  include GlobalID::Identification
  include Housekeeping

  include Shared::Citations
  include Shared::Containable
  include Shared::DataAttributes
  include Shared::Loanable
  include Shared::Identifiers
  include Shared::Notes
  include Shared::Tags
  include Shared::Depictions
  include Shared::OriginRelationship
  include Shared::Confidences
  include Shared::ProtocolRelationships
  include Shared::HasPapertrail
  include Shared::Observations
  include Shared::IsData
  include Shared::QueryBatchUpdate
  include SoftValidation

  include Shared::BiologicalExtensions

  include Shared::Taxonomy # at present must be before IsDwcOccurence
  include Shared::IsDwcOccurrence
  include CollectionObject::DwcExtensions

  ignore_whitespace_on(:buffered_collecting_event, :buffered_determinations, :buffered_other_labels)

  # TODO: move to export
  CO_OTU_HEADERS = %w{OTU OTU\ name Family Genus Species Country State County Locality Latitude Longitude}.freeze

  BUFFERED_ATTRIBUTES = %i{buffered_collecting_event buffered_determinations buffered_other_labels}.freeze

  GRAPH_ENTRY_POINTS = [:biological_associations, :data_attributes, :taxon_determinations, :biocuration_classifications, :collecting_event, :origin_relationships, :extracts, :observation_matrices]

  # Identifier delegations
  # .catalog_number_cached
  delegate :cached, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true
  # .catalog_number_namespace
  delegate :namespace, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true

  # .record_number_cached
  delegate :cached, to: :preferred_record_number, prefix: :record_number, allow_nil: true
  # .record_number_namespace
  delegate :namespace, to: :preferred_record_number, prefix: :record_number, allow_nil: true

  # CollectingEvent delegations
  delegate :map_center, to: :collecting_event, prefix: :collecting_event, allow_nil: true
  delegate :collectors, to: :collecting_event, prefix: :collecting_event, allow_nil: true

  # Repository delegations
  delegate :acronym, to: :repository, prefix: :repository, allow_nil: true
  delegate :url, to: :repository, prefix: :repository, allow_nil: true
  delegate :institutional_LSID, to: :repository, prefix: :repository, allow_nil: true

  # Preparation delegations
  delegate :name, to: :preparation_type, prefix: :preparation_type, allow_nil: true

  has_one :accession_provider_role, class_name: 'AccessionProvider', as: :role_object, dependent: :destroy
  has_one :accession_provider, through: :accession_provider_role, source: :person
  has_one :deaccession_recipient_role, class_name: 'DeaccessionRecipient', as: :role_object, dependent: :destroy
  has_one :deaccession_recipient, through: :deaccession_recipient_role, source: :person

  # TODO: Deprecate these models.  Semantics also confuse with origin relationship.
  has_many :derived_collection_objects, inverse_of: :collection_object, dependent: :restrict_with_error
  has_many :collection_object_observations, through: :derived_collection_objects, inverse_of: :collection_objects

  has_many :sqed_depictions, through: :depictions, dependent: :restrict_with_error

  belongs_to :collecting_event, inverse_of: :collection_objects
  belongs_to :preparation_type, inverse_of: :collection_objects
  belongs_to :ranged_lot_category, inverse_of: :ranged_lots
  belongs_to :repository, inverse_of: :collection_objects
  belongs_to :current_repository, class_name: 'Repository', inverse_of: :collection_objects

  has_many :georeferences, through: :collecting_event
  has_many :geographic_items, through: :georeferences

  has_many :collectors, through: :collecting_event

  accepts_nested_attributes_for :collecting_event, allow_destroy: true, reject_if: :reject_collecting_event

  before_validation :assign_type_if_total_or_ranged_lot_category_id_provided

  validates_presence_of :type
  validate :check_that_either_total_or_ranged_lot_category_id_is_present
  validate :check_that_both_of_category_and_total_are_not_present
  validate :collecting_event_belongs_to_project

  soft_validate(
    :sv_missing_accession_fields,
    set: :missing_accession_fields,
    name: 'Missing accession fields',
    description: 'Name or Provider are not selected')

  soft_validate(
    :sv_missing_deaccession_fields,
    set: :missing_deaccession_fields,
    name: 'Missing deaccesson fields',
    description: 'Date, recipient, or reason are not specified')

  scope :with_sequence_name, ->(name) { joins(sequence_join_hack_sql).where(sequences: {name:}) }
  scope :via_descriptor, ->(descriptor) { joins(sequence_join_hack_sql).where(sequences: {id: descriptor.sequences}) }

  has_many :extracts, through: :origin_relationships, source: :new_object, source_type: 'Extract'
  has_many :sequences, through: :extracts

  # This is a hack, maybe related to a Rails 5.1 bug.
  # It returns the SQL that works in 5.0/4.2 that
  # links CollectionObject to Sequences:
  # joins(derived_extracts: [:derived_sequences])
  def self.sequence_join_hack_sql
    %Q{INNER JOIN  "origin_relationships"
               ON  "origin_relationships"."old_object_id" = "collection_objects"."id"
                  AND  "origin_relationships"."new_object_type" = 'Extract'
                  AND  "origin_relationships"."old_object_type" = 'CollectionObject'
       INNER JOIN  "extracts"
               ON  "extracts"."id" =  "origin_relationships"."new_object_id"
       INNER JOIN  "origin_relationships" "origin_relationships_extracts_join"
               ON  "origin_relationships_extracts_join"."old_object_id" = "extracts"."id"
                  AND  "origin_relationships_extracts_join"."new_object_type" = 'Sequence'
                  AND  "origin_relationships_extracts_join"."old_object_type" = 'Extract'
       INNER JOIN  "sequences"
               ON  "sequences"."id" = "origin_relationships_extracts_join"."new_object_id"}
  end

  def self.batch_update(params)
    request = QueryBatchRequest.new(
      async_cutoff: params[:async_cutoff] || 50,
      klass: 'CollectionObject',
      object_filter_params: params[:collection_object_query],
      object_params: params[:collection_object],
      preview: params[:preview],
    )

    request.cap = 1000

    query_batch_update(request)
  end

  def self.batch_update_dwc_occurrence(params)
    q = Queries::CollectionObject::Filter.new(params).all

    r = BatchResponse.new
    r.method = 'batch_update_dwc_occurrence'
    r.klass = 'CollectionObject'

    c = q.all.count

    if c == 0 || c > 10000
      r.cap_reason = 'Too many (or no) collection objects (max 10k)'
      return r
    end

    if c < 51
      q.each do |co|
        co.set_dwc_occurrence
        r.updated.push co.id
      end
    else
      r.async = true
      q.each do |co|
        co.dwc_occurrence_update_query
      end
    end

    return r
  end

  def dwc_occurrence_update_query
    self.send(:set_dwc_occurrence)
  end

  handle_asynchronously :dwc_occurrence_update_query, run_at: Proc.new { 1.second.from_now }, queue: :query_batch_update

  # TODO: move to a helper
  def self.breakdown_status(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array

    breakdown = {
      total_objects:     collection_objects.length,
      collecting_events: {},
      determinations:    {},
      bio_overview:      []
    }

    breakdown.merge!(breakdown_buffered(collection_objects))

    collection_objects.each do |co|
      breakdown[:collecting_events].merge!(co => co.collecting_event) if co.collecting_event
      breakdown[:determinations].merge!(co => co.taxon_determinations) if co.taxon_determinations.load.any?
      breakdown[:bio_overview].push([co.total, co.biocuration_classes.collect { |a| a.name }])
    end

    breakdown
  end

  # @return [Hash]
  #   a unque list of buffered_ values observed in the collection objects passed
  def self.breakdown_buffered(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array
    breakdown = {}
    categories = BUFFERED_ATTRIBUTES

    categories.each do |c|
      breakdown[c] = []
    end

    categories.each do |c|
      collection_objects.each do |co|
        breakdown[c].push co.send(c)
      end
    end

    categories.each do |c|
      breakdown[c].uniq!
    end

    breakdown
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.earliest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:end_date_year)

    return EARLIEST_DATE if a.nil? && b.nil?  # 1700-01-01

    d = nil

    if a && b
      if a < b
        d = a
      end
    else
      d = a || b
    end
    d.to_s + '-01-01'
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.latest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:end_date_year)

    c = Time.now.strftime('%Y-%m-%d')

    return c if a.nil? && b.nil?

    d = nil

    if a && b
      if a > b
        d = a
      end
    else
      d = a || b
    end

    d.to_s + '/12/31'
  end

  # TODO: Clarify this.
  # CAREFULL - this isn't _in_, this is *with*, if it was in it would be spatial query, not a join(:geographic_items)
  #
  # Find all collection objects which have collecting events which have georeferences which have geographic_items which
  # are located within the geographic item supplied
  # @param [GeographicItem] geographic_item_id
  # @return [Scope] of CollectionObject
  def self.in_geographic_item(geographic_item, limit, steps = false)
    geographic_item_id = geographic_item.id
    if steps
      gi = GeographicItem.find(geographic_item_id)
      # find the geographic_items inside gi
      step_1 = GeographicItem.is_contained_by('any', gi) # .pluck(:id)
      # find the georeferences from the geographic_items
      step_2 = step_1.map(&:georeferences).uniq.flatten
      # find the collecting events connected to the georeferences
      step_3 = step_2.map(&:collecting_event).uniq.flatten
      # find the collection objects associated with the collecting events
      step_4 = step_3.map(&:collection_objects).flatten.map(&:id).uniq
      retval = CollectionObject.where(id: step_4.sort)
    else
      retval = CollectionObject.joins(:geographic_items)
        .where(GeographicItem.contained_by_where_sql(geographic_item.id))
        .limit(limit)
        .includes(:data_attributes, :collecting_event)
    end
    retval
  end

  # TODO: deprecate
  def self.selected_column_names
    @selected_column_names = {
      ce: {in: {}, im: {}},
      co: {in: {}, im: {}},
      bc: {in: {}, im: {}}
    } if @selected_column_names.nil?
    @selected_column_names
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collecting events
  # decode which headers to be displayed for collecting events
  def self.ce_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:ce][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:ce][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.ce_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.collecting_event.internal_attributes
      all_import_das   = collection_object.collecting_event.import_attributes
      group            = collection[:ce]
      unless group.nil?
        group.each_key { |type_key|
          group[type_key.to_sym].each_key { |header|
            this_val = nil
            case type_key.to_sym
            when :in
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            when :im
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            else
            end
          }
        }
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collection objects
  # decode which headers to be displayed for collection objects
  def self.co_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:co][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:co][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.co_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.internal_attributes
      all_import_das   = collection_object.import_attributes
      group            = collection[:co]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = nil
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
        unless group.empty?
          unless group[:im].empty?
            group[:im].each_key { |header|
              this_val = nil
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for biocuration classifications
  # decode which headers to be displayed for biocuration classifications
  def self.bc_headers(project_id)
    CollectionObject.selected_column_names
    # add selectable column names (unselected) to the column name list list
    BiocurationClass.where(project_id:).map(&:name).each { |column_name|
      @selected_column_names[:bc][:in][column_name] = {checked: '0'}
    }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.bc_attributes(collection_object, col_defs)
    retval = []
    collection = col_defs
    unless collection.nil?
      group = collection[:bc]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = collection_object.biocuration_classes.map(&:name).include?(header) ? '1' : '0'
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Array] collecting_event_ids (e.g., from CollectingEvent.in_date_range)
  # @param [Array] area_object_ids (e.g., from GeographicItem.gather_selected_data())
  # @return [Scope] of intersection of collecting events (usually by date range)
  #   and collection objects (usually by inclusion in geographic areas/items)
  def self.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id)
    collecting_events_clause = {collecting_event_id: collecting_event_ids, project: project_id}
    area_objects_clause      = {id: area_object_ids, project: project_id}

    if (collecting_event_ids.empty?)
      collecting_events_clause = {project: project_id}
    end

    if (area_object_ids.empty?)
      area_objects_clause = {}
      if (area_set)
        area_objects_clause = 'false'
      end
    end

    retval = CollectionObject.joins(:collecting_event)
      .where(collecting_events_clause)
      .where(area_objects_clause)
    retval
  end

  # TODO: move to filter
  # @param [Hash] search_start_date string in form 'yyyy-mm-dd'
  # @param [Hash] search_end_date string in form 'yyyy-mm-dd'
  # @param [Hash] partial_overlap 'on' or 'off'
  # @return [Scope] of selected collection objects through collecting events with georeferences, remember to scope to project!
  def self.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on')
    allow_partial = (partial_overlap.downcase == 'off' ? false : true) # TODO: Just get the correct values from the form!
    q = Queries::CollectingEvent::Filter.new(start_date: search_start_date, end_date: search_end_date, partial_overlap_dates: allow_partial)
    joins(:collecting_event).where(q.between_date_range_facet.to_sql)
  end

  # @param used_on [String] required, one of `TaxonDetermination`, `BiologicalAssociation`
  # @return [Scope]
  #    the max 10 most recently used collection_objects, as `used_on`
  def self.used_recently(user_id, project_id, used_on = '')
    return [] if used_on != 'TaxonDetermination' && used_on != 'BiologicalAssociation'
    t = case used_on
        when 'TaxonDetermination'
          TaxonDetermination.arel_table
        when 'BiologicalAssociation'
          BiologicalAssociation.arel_table
        end

    p = CollectionObject.arel_table

    # i is a select manager
    i = case used_on
        when 'BiologicalAssociation'
          t.project(t['biological_association_subject_id'], t['updated_at']).from(t)
            .where(
              t['updated_at'].gt(1.week.ago).and(
                t['biological_association_subject_type'].eq('CollectionObject')
              )
            )
              .where(t['updated_by_id'].eq(user_id))
              .where(t['project_id'].eq(project_id))
              .order(t['updated_at'].desc)
        else
          # TODO: update to reference new TaxonDetermination
          t.project(t['taxon_determination_object_id'], t['taxon_determination_object_type'], t['updated_at']).from(t)
            .where(t['updated_at'].gt( 1.week.ago ))
            .where(t['updated_by_id'].eq(user_id))
            .where(t['project_id'].eq(project_id))
            .order(t['updated_at'].desc)
        end

    # z is a table alias
    z = i.as('recent_t')

    j = case used_on
        when 'BiologicalAssociation'
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(
            z['biological_association_subject_id'].eq(p['id'])
          ))
        else
          # TODO: needs to be fixed to scope the taxon_determination_object_type
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(z['taxon_determination_object_id'].eq(p['id'])))
        end

    CollectionObject.joins(j).pluck(:id).uniq
  end

  # @params target [String] one of `TaxonDetermination`, `BiologicalAssociation` , nil
  # @return [Hash] otus optimized for user selection
  def self.select_optimized(user_id, project_id, target = nil)
    r = used_recently(user_id, project_id, target)
    h = {
      quick: [],
      pinboard: CollectionObject.pinned_by(user_id).where(project_id:).to_a,
      recent: []
    }

    if target && !r.empty?
      n = target.tableize.to_sym
      h[:recent] = CollectionObject.where('"collection_objects"."id" IN (?)', r.first(10) ).to_a
      h[:quick] = (CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a  +
                   CollectionObject.where('"collection_objects"."id" IN (?)', r.first(4) ).to_a).uniq
    else
      h[:recent] = CollectionObject.where(project_id:, updated_by_id: user_id).order('updated_at DESC').limit(10).to_a
      h[:quick] = CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a
    end

    h
  end

  # TODO: Unify with Extract in concern
  # @return [Identifier::Local::CatalogNumber, nil]
  #   the first (position) catalog number for this collection object, either on specimen, or container
  def preferred_catalog_number
    if i = Identifier::Local::CatalogNumber.where(identifier_object: self).order(:position).first
      i
    else
      if container
        container.identifiers.where(identifiers: {type: 'Identifier::Local::CatalogNumber'}).order(:position).first
      else
        nil
      end
    end
  end

  # @return [Identifier::Local::RecordNumber, nil]
  #   the first (position) record_Number, on a specimen
  #   !1 Doesn't presently support containers
  def preferred_record_number
    Identifier::Local::RecordNumber.where(identifier_object: self).order(:position).first
  end

  def geographic_name_classification
    # don't load the whole object, just the fields we need
    if a = DwcOccurrence.where(dwc_occurrence_object: self).select(:country, :stateProvince, :county).first

      c = a.country
      s = a.stateProvince
      y = a.county

      v = ::Utilities::Geo::DICTIONARY[c]
      c = v if v
      # s = v if v = ::Utilities::Geo::DICTIONARY[s] # None in there yet
      # y = v if v = ::Utilities::Geo::DICTIONARY[y] # None in there yet

      return {
        country: c,
        state: s,
        county: y
      }
    end
  end

  # return [Boolean]
  #    True if instance is a subclass of BiologicalCollectionObject
  def is_biological?
    self.class <= BiologicalCollectionObject ? true : false
  end

  def annotations
    h = annotations_hash
    (h['biocuration classifications'] = biocuration_classes) if is_biological? && biocuration_classifications.load.any?
    h
  end

  def sv_missing_accession_fields
    soft_validations.add(:accessioned_at, 'Date is not selected') if self.accessioned_at.nil? && !self.accession_provider.nil?
    soft_validations.add(:base, 'Provider is not selected') if !self.accessioned_at.nil? && self.accession_provider.nil?
  end

  def sv_missing_deaccession_fields
    soft_validations.add(:deaccessioned_at, 'Date is not selected') if self.deaccessioned_at.nil? && self.deaccession_reason.present?
    soft_validations.add(:base, 'Recipient is not selected') if self.deaccession_recipient.nil? && self.deaccession_reason && self.deaccessioned_at
    soft_validations.add(:deaccession_reason, 'Reason is is not defined') if self.deaccession_reason.blank? && self.deaccession_recipient && self.deaccessioned_at
  end

  def sv_missing_determination
    # see biological_collection_object
  end

  def sv_missing_collecting_event
    # see biological_collection_object
  end

  def sv_missing_preparation_type
    # see biological_collection_object
  end

  def sv_missing_repository
    # WHY? -  see biological_collection_object
  end

  def sv_missing_biocuration_classification
    # see biological_collection_object
  end

  # See Depiction#destroy_image_stub_collection_object
  # Used to determin if the CO can be
  # destroy after moving an image off
  # this object.
  def is_image_stub?
    r = [
      collecting_event_id.blank?,
      !depictions.reload.any?,
      identifiers.count <= 1,
      !taxon_determinations.any?,
      !type_materials.any?,
      !citations.any?,
      !data_attributes.any?,
      !notes.any?,
      !observations.any?
    ]

   !r.include?(false)

  end

  protected

  def collecting_event_belongs_to_project
    if collecting_event&.persisted? && (Current.project_id || project_id)
      errors.add(:base, 'collecting event is not from this project') if collecting_event.project_id != (Current.project_id || project_id)
    end
  end

  def check_that_both_of_category_and_total_are_not_present
    errors.add(:ranged_lot_category_id, 'Both ranged_lot_category and total can not be set') if ranged_lot_category_id.present? && total.present?
  end

  def check_that_either_total_or_ranged_lot_category_id_is_present
    errors.add(:base, 'Either total or a ranged lot category must be provided') if ranged_lot_category_id.blank? && total.blank?
  end

  def assign_type_if_total_or_ranged_lot_category_id_provided
    if self.total == 1
      self.type = 'Specimen'
    elsif self.total.to_i > 1
      self.type = 'Lot'
    elsif total.nil? && ranged_lot_category_id.present?
      self.type = 'RangedLot'
    end
    true
  end

  def reject_collecting_event(attributed)
    reject = true
    CollectingEvent.core_attributes.each do |a|
      if attributed[a].present?
        reject = false
        break
      end
    end
    # !! does not account for georeferences_attributes!
    reject
  end

end

#buffered_collecting_event ⇒ String

An incoming, typically verbatim, block of data typically as seens as a locality/method/etc. label. All buffered_ attributes are written but not intended to be deleted or otherwise updated. Buffered_ attributes are typically only used in rapid data capture, primarily in historical situations.

Returns:

• (String)

# File 'app/models/collection_object.rb', line 64

class CollectionObject < ApplicationRecord
  include GlobalID::Identification
  include Housekeeping

  include Shared::Citations
  include Shared::Containable
  include Shared::DataAttributes
  include Shared::Loanable
  include Shared::Identifiers
  include Shared::Notes
  include Shared::Tags
  include Shared::Depictions
  include Shared::OriginRelationship
  include Shared::Confidences
  include Shared::ProtocolRelationships
  include Shared::HasPapertrail
  include Shared::Observations
  include Shared::IsData
  include Shared::QueryBatchUpdate
  include SoftValidation

  include Shared::BiologicalExtensions

  include Shared::Taxonomy # at present must be before IsDwcOccurence
  include Shared::IsDwcOccurrence
  include CollectionObject::DwcExtensions

  ignore_whitespace_on(:buffered_collecting_event, :buffered_determinations, :buffered_other_labels)

  # TODO: move to export
  CO_OTU_HEADERS = %w{OTU OTU\ name Family Genus Species Country State County Locality Latitude Longitude}.freeze

  BUFFERED_ATTRIBUTES = %i{buffered_collecting_event buffered_determinations buffered_other_labels}.freeze

  GRAPH_ENTRY_POINTS = [:biological_associations, :data_attributes, :taxon_determinations, :biocuration_classifications, :collecting_event, :origin_relationships, :extracts, :observation_matrices]

  # Identifier delegations
  # .catalog_number_cached
  delegate :cached, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true
  # .catalog_number_namespace
  delegate :namespace, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true

  # .record_number_cached
  delegate :cached, to: :preferred_record_number, prefix: :record_number, allow_nil: true
  # .record_number_namespace
  delegate :namespace, to: :preferred_record_number, prefix: :record_number, allow_nil: true

  # CollectingEvent delegations
  delegate :map_center, to: :collecting_event, prefix: :collecting_event, allow_nil: true
  delegate :collectors, to: :collecting_event, prefix: :collecting_event, allow_nil: true

  # Repository delegations
  delegate :acronym, to: :repository, prefix: :repository, allow_nil: true
  delegate :url, to: :repository, prefix: :repository, allow_nil: true
  delegate :institutional_LSID, to: :repository, prefix: :repository, allow_nil: true

  # Preparation delegations
  delegate :name, to: :preparation_type, prefix: :preparation_type, allow_nil: true

  has_one :accession_provider_role, class_name: 'AccessionProvider', as: :role_object, dependent: :destroy
  has_one :accession_provider, through: :accession_provider_role, source: :person
  has_one :deaccession_recipient_role, class_name: 'DeaccessionRecipient', as: :role_object, dependent: :destroy
  has_one :deaccession_recipient, through: :deaccession_recipient_role, source: :person

  # TODO: Deprecate these models.  Semantics also confuse with origin relationship.
  has_many :derived_collection_objects, inverse_of: :collection_object, dependent: :restrict_with_error
  has_many :collection_object_observations, through: :derived_collection_objects, inverse_of: :collection_objects

  has_many :sqed_depictions, through: :depictions, dependent: :restrict_with_error

  belongs_to :collecting_event, inverse_of: :collection_objects
  belongs_to :preparation_type, inverse_of: :collection_objects
  belongs_to :ranged_lot_category, inverse_of: :ranged_lots
  belongs_to :repository, inverse_of: :collection_objects
  belongs_to :current_repository, class_name: 'Repository', inverse_of: :collection_objects

  has_many :georeferences, through: :collecting_event
  has_many :geographic_items, through: :georeferences

  has_many :collectors, through: :collecting_event

  accepts_nested_attributes_for :collecting_event, allow_destroy: true, reject_if: :reject_collecting_event

  before_validation :assign_type_if_total_or_ranged_lot_category_id_provided

  validates_presence_of :type
  validate :check_that_either_total_or_ranged_lot_category_id_is_present
  validate :check_that_both_of_category_and_total_are_not_present
  validate :collecting_event_belongs_to_project

  soft_validate(
    :sv_missing_accession_fields,
    set: :missing_accession_fields,
    name: 'Missing accession fields',
    description: 'Name or Provider are not selected')

  soft_validate(
    :sv_missing_deaccession_fields,
    set: :missing_deaccession_fields,
    name: 'Missing deaccesson fields',
    description: 'Date, recipient, or reason are not specified')

  scope :with_sequence_name, ->(name) { joins(sequence_join_hack_sql).where(sequences: {name:}) }
  scope :via_descriptor, ->(descriptor) { joins(sequence_join_hack_sql).where(sequences: {id: descriptor.sequences}) }

  has_many :extracts, through: :origin_relationships, source: :new_object, source_type: 'Extract'
  has_many :sequences, through: :extracts

  # This is a hack, maybe related to a Rails 5.1 bug.
  # It returns the SQL that works in 5.0/4.2 that
  # links CollectionObject to Sequences:
  # joins(derived_extracts: [:derived_sequences])
  def self.sequence_join_hack_sql
    %Q{INNER JOIN  "origin_relationships"
               ON  "origin_relationships"."old_object_id" = "collection_objects"."id"
                  AND  "origin_relationships"."new_object_type" = 'Extract'
                  AND  "origin_relationships"."old_object_type" = 'CollectionObject'
       INNER JOIN  "extracts"
               ON  "extracts"."id" =  "origin_relationships"."new_object_id"
       INNER JOIN  "origin_relationships" "origin_relationships_extracts_join"
               ON  "origin_relationships_extracts_join"."old_object_id" = "extracts"."id"
                  AND  "origin_relationships_extracts_join"."new_object_type" = 'Sequence'
                  AND  "origin_relationships_extracts_join"."old_object_type" = 'Extract'
       INNER JOIN  "sequences"
               ON  "sequences"."id" = "origin_relationships_extracts_join"."new_object_id"}
  end

  def self.batch_update(params)
    request = QueryBatchRequest.new(
      async_cutoff: params[:async_cutoff] || 50,
      klass: 'CollectionObject',
      object_filter_params: params[:collection_object_query],
      object_params: params[:collection_object],
      preview: params[:preview],
    )

    request.cap = 1000

    query_batch_update(request)
  end

  def self.batch_update_dwc_occurrence(params)
    q = Queries::CollectionObject::Filter.new(params).all

    r = BatchResponse.new
    r.method = 'batch_update_dwc_occurrence'
    r.klass = 'CollectionObject'

    c = q.all.count

    if c == 0 || c > 10000
      r.cap_reason = 'Too many (or no) collection objects (max 10k)'
      return r
    end

    if c < 51
      q.each do |co|
        co.set_dwc_occurrence
        r.updated.push co.id
      end
    else
      r.async = true
      q.each do |co|
        co.dwc_occurrence_update_query
      end
    end

    return r
  end

  def dwc_occurrence_update_query
    self.send(:set_dwc_occurrence)
  end

  handle_asynchronously :dwc_occurrence_update_query, run_at: Proc.new { 1.second.from_now }, queue: :query_batch_update

  # TODO: move to a helper
  def self.breakdown_status(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array

    breakdown = {
      total_objects:     collection_objects.length,
      collecting_events: {},
      determinations:    {},
      bio_overview:      []
    }

    breakdown.merge!(breakdown_buffered(collection_objects))

    collection_objects.each do |co|
      breakdown[:collecting_events].merge!(co => co.collecting_event) if co.collecting_event
      breakdown[:determinations].merge!(co => co.taxon_determinations) if co.taxon_determinations.load.any?
      breakdown[:bio_overview].push([co.total, co.biocuration_classes.collect { |a| a.name }])
    end

    breakdown
  end

  # @return [Hash]
  #   a unque list of buffered_ values observed in the collection objects passed
  def self.breakdown_buffered(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array
    breakdown = {}
    categories = BUFFERED_ATTRIBUTES

    categories.each do |c|
      breakdown[c] = []
    end

    categories.each do |c|
      collection_objects.each do |co|
        breakdown[c].push co.send(c)
      end
    end

    categories.each do |c|
      breakdown[c].uniq!
    end

    breakdown
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.earliest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:end_date_year)

    return EARLIEST_DATE if a.nil? && b.nil?  # 1700-01-01

    d = nil

    if a && b
      if a < b
        d = a
      end
    else
      d = a || b
    end
    d.to_s + '-01-01'
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.latest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:end_date_year)

    c = Time.now.strftime('%Y-%m-%d')

    return c if a.nil? && b.nil?

    d = nil

    if a && b
      if a > b
        d = a
      end
    else
      d = a || b
    end

    d.to_s + '/12/31'
  end

  # TODO: Clarify this.
  # CAREFULL - this isn't _in_, this is *with*, if it was in it would be spatial query, not a join(:geographic_items)
  #
  # Find all collection objects which have collecting events which have georeferences which have geographic_items which
  # are located within the geographic item supplied
  # @param [GeographicItem] geographic_item_id
  # @return [Scope] of CollectionObject
  def self.in_geographic_item(geographic_item, limit, steps = false)
    geographic_item_id = geographic_item.id
    if steps
      gi = GeographicItem.find(geographic_item_id)
      # find the geographic_items inside gi
      step_1 = GeographicItem.is_contained_by('any', gi) # .pluck(:id)
      # find the georeferences from the geographic_items
      step_2 = step_1.map(&:georeferences).uniq.flatten
      # find the collecting events connected to the georeferences
      step_3 = step_2.map(&:collecting_event).uniq.flatten
      # find the collection objects associated with the collecting events
      step_4 = step_3.map(&:collection_objects).flatten.map(&:id).uniq
      retval = CollectionObject.where(id: step_4.sort)
    else
      retval = CollectionObject.joins(:geographic_items)
        .where(GeographicItem.contained_by_where_sql(geographic_item.id))
        .limit(limit)
        .includes(:data_attributes, :collecting_event)
    end
    retval
  end

  # TODO: deprecate
  def self.selected_column_names
    @selected_column_names = {
      ce: {in: {}, im: {}},
      co: {in: {}, im: {}},
      bc: {in: {}, im: {}}
    } if @selected_column_names.nil?
    @selected_column_names
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collecting events
  # decode which headers to be displayed for collecting events
  def self.ce_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:ce][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:ce][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.ce_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.collecting_event.internal_attributes
      all_import_das   = collection_object.collecting_event.import_attributes
      group            = collection[:ce]
      unless group.nil?
        group.each_key { |type_key|
          group[type_key.to_sym].each_key { |header|
            this_val = nil
            case type_key.to_sym
            when :in
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            when :im
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            else
            end
          }
        }
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collection objects
  # decode which headers to be displayed for collection objects
  def self.co_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:co][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:co][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.co_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.internal_attributes
      all_import_das   = collection_object.import_attributes
      group            = collection[:co]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = nil
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
        unless group.empty?
          unless group[:im].empty?
            group[:im].each_key { |header|
              this_val = nil
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for biocuration classifications
  # decode which headers to be displayed for biocuration classifications
  def self.bc_headers(project_id)
    CollectionObject.selected_column_names
    # add selectable column names (unselected) to the column name list list
    BiocurationClass.where(project_id:).map(&:name).each { |column_name|
      @selected_column_names[:bc][:in][column_name] = {checked: '0'}
    }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.bc_attributes(collection_object, col_defs)
    retval = []
    collection = col_defs
    unless collection.nil?
      group = collection[:bc]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = collection_object.biocuration_classes.map(&:name).include?(header) ? '1' : '0'
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Array] collecting_event_ids (e.g., from CollectingEvent.in_date_range)
  # @param [Array] area_object_ids (e.g., from GeographicItem.gather_selected_data())
  # @return [Scope] of intersection of collecting events (usually by date range)
  #   and collection objects (usually by inclusion in geographic areas/items)
  def self.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id)
    collecting_events_clause = {collecting_event_id: collecting_event_ids, project: project_id}
    area_objects_clause      = {id: area_object_ids, project: project_id}

    if (collecting_event_ids.empty?)
      collecting_events_clause = {project: project_id}
    end

    if (area_object_ids.empty?)
      area_objects_clause = {}
      if (area_set)
        area_objects_clause = 'false'
      end
    end

    retval = CollectionObject.joins(:collecting_event)
      .where(collecting_events_clause)
      .where(area_objects_clause)
    retval
  end

  # TODO: move to filter
  # @param [Hash] search_start_date string in form 'yyyy-mm-dd'
  # @param [Hash] search_end_date string in form 'yyyy-mm-dd'
  # @param [Hash] partial_overlap 'on' or 'off'
  # @return [Scope] of selected collection objects through collecting events with georeferences, remember to scope to project!
  def self.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on')
    allow_partial = (partial_overlap.downcase == 'off' ? false : true) # TODO: Just get the correct values from the form!
    q = Queries::CollectingEvent::Filter.new(start_date: search_start_date, end_date: search_end_date, partial_overlap_dates: allow_partial)
    joins(:collecting_event).where(q.between_date_range_facet.to_sql)
  end

  # @param used_on [String] required, one of `TaxonDetermination`, `BiologicalAssociation`
  # @return [Scope]
  #    the max 10 most recently used collection_objects, as `used_on`
  def self.used_recently(user_id, project_id, used_on = '')
    return [] if used_on != 'TaxonDetermination' && used_on != 'BiologicalAssociation'
    t = case used_on
        when 'TaxonDetermination'
          TaxonDetermination.arel_table
        when 'BiologicalAssociation'
          BiologicalAssociation.arel_table
        end

    p = CollectionObject.arel_table

    # i is a select manager
    i = case used_on
        when 'BiologicalAssociation'
          t.project(t['biological_association_subject_id'], t['updated_at']).from(t)
            .where(
              t['updated_at'].gt(1.week.ago).and(
                t['biological_association_subject_type'].eq('CollectionObject')
              )
            )
              .where(t['updated_by_id'].eq(user_id))
              .where(t['project_id'].eq(project_id))
              .order(t['updated_at'].desc)
        else
          # TODO: update to reference new TaxonDetermination
          t.project(t['taxon_determination_object_id'], t['taxon_determination_object_type'], t['updated_at']).from(t)
            .where(t['updated_at'].gt( 1.week.ago ))
            .where(t['updated_by_id'].eq(user_id))
            .where(t['project_id'].eq(project_id))
            .order(t['updated_at'].desc)
        end

    # z is a table alias
    z = i.as('recent_t')

    j = case used_on
        when 'BiologicalAssociation'
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(
            z['biological_association_subject_id'].eq(p['id'])
          ))
        else
          # TODO: needs to be fixed to scope the taxon_determination_object_type
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(z['taxon_determination_object_id'].eq(p['id'])))
        end

    CollectionObject.joins(j).pluck(:id).uniq
  end

  # @params target [String] one of `TaxonDetermination`, `BiologicalAssociation` , nil
  # @return [Hash] otus optimized for user selection
  def self.select_optimized(user_id, project_id, target = nil)
    r = used_recently(user_id, project_id, target)
    h = {
      quick: [],
      pinboard: CollectionObject.pinned_by(user_id).where(project_id:).to_a,
      recent: []
    }

    if target && !r.empty?
      n = target.tableize.to_sym
      h[:recent] = CollectionObject.where('"collection_objects"."id" IN (?)', r.first(10) ).to_a
      h[:quick] = (CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a  +
                   CollectionObject.where('"collection_objects"."id" IN (?)', r.first(4) ).to_a).uniq
    else
      h[:recent] = CollectionObject.where(project_id:, updated_by_id: user_id).order('updated_at DESC').limit(10).to_a
      h[:quick] = CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a
    end

    h
  end

  # TODO: Unify with Extract in concern
  # @return [Identifier::Local::CatalogNumber, nil]
  #   the first (position) catalog number for this collection object, either on specimen, or container
  def preferred_catalog_number
    if i = Identifier::Local::CatalogNumber.where(identifier_object: self).order(:position).first
      i
    else
      if container
        container.identifiers.where(identifiers: {type: 'Identifier::Local::CatalogNumber'}).order(:position).first
      else
        nil
      end
    end
  end

  # @return [Identifier::Local::RecordNumber, nil]
  #   the first (position) record_Number, on a specimen
  #   !1 Doesn't presently support containers
  def preferred_record_number
    Identifier::Local::RecordNumber.where(identifier_object: self).order(:position).first
  end

  def geographic_name_classification
    # don't load the whole object, just the fields we need
    if a = DwcOccurrence.where(dwc_occurrence_object: self).select(:country, :stateProvince, :county).first

      c = a.country
      s = a.stateProvince
      y = a.county

      v = ::Utilities::Geo::DICTIONARY[c]
      c = v if v
      # s = v if v = ::Utilities::Geo::DICTIONARY[s] # None in there yet
      # y = v if v = ::Utilities::Geo::DICTIONARY[y] # None in there yet

      return {
        country: c,
        state: s,
        county: y
      }
    end
  end

  # return [Boolean]
  #    True if instance is a subclass of BiologicalCollectionObject
  def is_biological?
    self.class <= BiologicalCollectionObject ? true : false
  end

  def annotations
    h = annotations_hash
    (h['biocuration classifications'] = biocuration_classes) if is_biological? && biocuration_classifications.load.any?
    h
  end

  def sv_missing_accession_fields
    soft_validations.add(:accessioned_at, 'Date is not selected') if self.accessioned_at.nil? && !self.accession_provider.nil?
    soft_validations.add(:base, 'Provider is not selected') if !self.accessioned_at.nil? && self.accession_provider.nil?
  end

  def sv_missing_deaccession_fields
    soft_validations.add(:deaccessioned_at, 'Date is not selected') if self.deaccessioned_at.nil? && self.deaccession_reason.present?
    soft_validations.add(:base, 'Recipient is not selected') if self.deaccession_recipient.nil? && self.deaccession_reason && self.deaccessioned_at
    soft_validations.add(:deaccession_reason, 'Reason is is not defined') if self.deaccession_reason.blank? && self.deaccession_recipient && self.deaccessioned_at
  end

  def sv_missing_determination
    # see biological_collection_object
  end

  def sv_missing_collecting_event
    # see biological_collection_object
  end

  def sv_missing_preparation_type
    # see biological_collection_object
  end

  def sv_missing_repository
    # WHY? -  see biological_collection_object
  end

  def sv_missing_biocuration_classification
    # see biological_collection_object
  end

  # See Depiction#destroy_image_stub_collection_object
  # Used to determin if the CO can be
  # destroy after moving an image off
  # this object.
  def is_image_stub?
    r = [
      collecting_event_id.blank?,
      !depictions.reload.any?,
      identifiers.count <= 1,
      !taxon_determinations.any?,
      !type_materials.any?,
      !citations.any?,
      !data_attributes.any?,
      !notes.any?,
      !observations.any?
    ]

   !r.include?(false)

  end

  protected

  def collecting_event_belongs_to_project
    if collecting_event&.persisted? && (Current.project_id || project_id)
      errors.add(:base, 'collecting event is not from this project') if collecting_event.project_id != (Current.project_id || project_id)
    end
  end

  def check_that_both_of_category_and_total_are_not_present
    errors.add(:ranged_lot_category_id, 'Both ranged_lot_category and total can not be set') if ranged_lot_category_id.present? && total.present?
  end

  def check_that_either_total_or_ranged_lot_category_id_is_present
    errors.add(:base, 'Either total or a ranged lot category must be provided') if ranged_lot_category_id.blank? && total.blank?
  end

  def assign_type_if_total_or_ranged_lot_category_id_provided
    if self.total == 1
      self.type = 'Specimen'
    elsif self.total.to_i > 1
      self.type = 'Lot'
    elsif total.nil? && ranged_lot_category_id.present?
      self.type = 'RangedLot'
    end
    true
  end

  def reject_collecting_event(attributed)
    reject = true
    CollectingEvent.core_attributes.each do |a|
      if attributed[a].present?
        reject = false
        break
      end
    end
    # !! does not account for georeferences_attributes!
    reject
  end

end

#buffered_determinations ⇒ String

An incoming, typically verbatim, block of data typically as seen a taxonomic determination label. All buffered_ attributes are written but not intended to be deleted or otherwise updated. Buffered_ attributes are typically only used in rapid data capture, primarily in historical situations.

Returns:

• (String)

# File 'app/models/collection_object.rb', line 64

class CollectionObject < ApplicationRecord
  include GlobalID::Identification
  include Housekeeping

  include Shared::Citations
  include Shared::Containable
  include Shared::DataAttributes
  include Shared::Loanable
  include Shared::Identifiers
  include Shared::Notes
  include Shared::Tags
  include Shared::Depictions
  include Shared::OriginRelationship
  include Shared::Confidences
  include Shared::ProtocolRelationships
  include Shared::HasPapertrail
  include Shared::Observations
  include Shared::IsData
  include Shared::QueryBatchUpdate
  include SoftValidation

  include Shared::BiologicalExtensions

  include Shared::Taxonomy # at present must be before IsDwcOccurence
  include Shared::IsDwcOccurrence
  include CollectionObject::DwcExtensions

  ignore_whitespace_on(:buffered_collecting_event, :buffered_determinations, :buffered_other_labels)

  # TODO: move to export
  CO_OTU_HEADERS = %w{OTU OTU\ name Family Genus Species Country State County Locality Latitude Longitude}.freeze

  BUFFERED_ATTRIBUTES = %i{buffered_collecting_event buffered_determinations buffered_other_labels}.freeze

  GRAPH_ENTRY_POINTS = [:biological_associations, :data_attributes, :taxon_determinations, :biocuration_classifications, :collecting_event, :origin_relationships, :extracts, :observation_matrices]

  # Identifier delegations
  # .catalog_number_cached
  delegate :cached, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true
  # .catalog_number_namespace
  delegate :namespace, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true

  # .record_number_cached
  delegate :cached, to: :preferred_record_number, prefix: :record_number, allow_nil: true
  # .record_number_namespace
  delegate :namespace, to: :preferred_record_number, prefix: :record_number, allow_nil: true

  # CollectingEvent delegations
  delegate :map_center, to: :collecting_event, prefix: :collecting_event, allow_nil: true
  delegate :collectors, to: :collecting_event, prefix: :collecting_event, allow_nil: true

  # Repository delegations
  delegate :acronym, to: :repository, prefix: :repository, allow_nil: true
  delegate :url, to: :repository, prefix: :repository, allow_nil: true
  delegate :institutional_LSID, to: :repository, prefix: :repository, allow_nil: true

  # Preparation delegations
  delegate :name, to: :preparation_type, prefix: :preparation_type, allow_nil: true

  has_one :accession_provider_role, class_name: 'AccessionProvider', as: :role_object, dependent: :destroy
  has_one :accession_provider, through: :accession_provider_role, source: :person
  has_one :deaccession_recipient_role, class_name: 'DeaccessionRecipient', as: :role_object, dependent: :destroy
  has_one :deaccession_recipient, through: :deaccession_recipient_role, source: :person

  # TODO: Deprecate these models.  Semantics also confuse with origin relationship.
  has_many :derived_collection_objects, inverse_of: :collection_object, dependent: :restrict_with_error
  has_many :collection_object_observations, through: :derived_collection_objects, inverse_of: :collection_objects

  has_many :sqed_depictions, through: :depictions, dependent: :restrict_with_error

  belongs_to :collecting_event, inverse_of: :collection_objects
  belongs_to :preparation_type, inverse_of: :collection_objects
  belongs_to :ranged_lot_category, inverse_of: :ranged_lots
  belongs_to :repository, inverse_of: :collection_objects
  belongs_to :current_repository, class_name: 'Repository', inverse_of: :collection_objects

  has_many :georeferences, through: :collecting_event
  has_many :geographic_items, through: :georeferences

  has_many :collectors, through: :collecting_event

  accepts_nested_attributes_for :collecting_event, allow_destroy: true, reject_if: :reject_collecting_event

  before_validation :assign_type_if_total_or_ranged_lot_category_id_provided

  validates_presence_of :type
  validate :check_that_either_total_or_ranged_lot_category_id_is_present
  validate :check_that_both_of_category_and_total_are_not_present
  validate :collecting_event_belongs_to_project

  soft_validate(
    :sv_missing_accession_fields,
    set: :missing_accession_fields,
    name: 'Missing accession fields',
    description: 'Name or Provider are not selected')

  soft_validate(
    :sv_missing_deaccession_fields,
    set: :missing_deaccession_fields,
    name: 'Missing deaccesson fields',
    description: 'Date, recipient, or reason are not specified')

  scope :with_sequence_name, ->(name) { joins(sequence_join_hack_sql).where(sequences: {name:}) }
  scope :via_descriptor, ->(descriptor) { joins(sequence_join_hack_sql).where(sequences: {id: descriptor.sequences}) }

  has_many :extracts, through: :origin_relationships, source: :new_object, source_type: 'Extract'
  has_many :sequences, through: :extracts

  # This is a hack, maybe related to a Rails 5.1 bug.
  # It returns the SQL that works in 5.0/4.2 that
  # links CollectionObject to Sequences:
  # joins(derived_extracts: [:derived_sequences])
  def self.sequence_join_hack_sql
    %Q{INNER JOIN  "origin_relationships"
               ON  "origin_relationships"."old_object_id" = "collection_objects"."id"
                  AND  "origin_relationships"."new_object_type" = 'Extract'
                  AND  "origin_relationships"."old_object_type" = 'CollectionObject'
       INNER JOIN  "extracts"
               ON  "extracts"."id" =  "origin_relationships"."new_object_id"
       INNER JOIN  "origin_relationships" "origin_relationships_extracts_join"
               ON  "origin_relationships_extracts_join"."old_object_id" = "extracts"."id"
                  AND  "origin_relationships_extracts_join"."new_object_type" = 'Sequence'
                  AND  "origin_relationships_extracts_join"."old_object_type" = 'Extract'
       INNER JOIN  "sequences"
               ON  "sequences"."id" = "origin_relationships_extracts_join"."new_object_id"}
  end

  def self.batch_update(params)
    request = QueryBatchRequest.new(
      async_cutoff: params[:async_cutoff] || 50,
      klass: 'CollectionObject',
      object_filter_params: params[:collection_object_query],
      object_params: params[:collection_object],
      preview: params[:preview],
    )

    request.cap = 1000

    query_batch_update(request)
  end

  def self.batch_update_dwc_occurrence(params)
    q = Queries::CollectionObject::Filter.new(params).all

    r = BatchResponse.new
    r.method = 'batch_update_dwc_occurrence'
    r.klass = 'CollectionObject'

    c = q.all.count

    if c == 0 || c > 10000
      r.cap_reason = 'Too many (or no) collection objects (max 10k)'
      return r
    end

    if c < 51
      q.each do |co|
        co.set_dwc_occurrence
        r.updated.push co.id
      end
    else
      r.async = true
      q.each do |co|
        co.dwc_occurrence_update_query
      end
    end

    return r
  end

  def dwc_occurrence_update_query
    self.send(:set_dwc_occurrence)
  end

  handle_asynchronously :dwc_occurrence_update_query, run_at: Proc.new { 1.second.from_now }, queue: :query_batch_update

  # TODO: move to a helper
  def self.breakdown_status(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array

    breakdown = {
      total_objects:     collection_objects.length,
      collecting_events: {},
      determinations:    {},
      bio_overview:      []
    }

    breakdown.merge!(breakdown_buffered(collection_objects))

    collection_objects.each do |co|
      breakdown[:collecting_events].merge!(co => co.collecting_event) if co.collecting_event
      breakdown[:determinations].merge!(co => co.taxon_determinations) if co.taxon_determinations.load.any?
      breakdown[:bio_overview].push([co.total, co.biocuration_classes.collect { |a| a.name }])
    end

    breakdown
  end

  # @return [Hash]
  #   a unque list of buffered_ values observed in the collection objects passed
  def self.breakdown_buffered(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array
    breakdown = {}
    categories = BUFFERED_ATTRIBUTES

    categories.each do |c|
      breakdown[c] = []
    end

    categories.each do |c|
      collection_objects.each do |co|
        breakdown[c].push co.send(c)
      end
    end

    categories.each do |c|
      breakdown[c].uniq!
    end

    breakdown
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.earliest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:end_date_year)

    return EARLIEST_DATE if a.nil? && b.nil?  # 1700-01-01

    d = nil

    if a && b
      if a < b
        d = a
      end
    else
      d = a || b
    end
    d.to_s + '-01-01'
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.latest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:end_date_year)

    c = Time.now.strftime('%Y-%m-%d')

    return c if a.nil? && b.nil?

    d = nil

    if a && b
      if a > b
        d = a
      end
    else
      d = a || b
    end

    d.to_s + '/12/31'
  end

  # TODO: Clarify this.
  # CAREFULL - this isn't _in_, this is *with*, if it was in it would be spatial query, not a join(:geographic_items)
  #
  # Find all collection objects which have collecting events which have georeferences which have geographic_items which
  # are located within the geographic item supplied
  # @param [GeographicItem] geographic_item_id
  # @return [Scope] of CollectionObject
  def self.in_geographic_item(geographic_item, limit, steps = false)
    geographic_item_id = geographic_item.id
    if steps
      gi = GeographicItem.find(geographic_item_id)
      # find the geographic_items inside gi
      step_1 = GeographicItem.is_contained_by('any', gi) # .pluck(:id)
      # find the georeferences from the geographic_items
      step_2 = step_1.map(&:georeferences).uniq.flatten
      # find the collecting events connected to the georeferences
      step_3 = step_2.map(&:collecting_event).uniq.flatten
      # find the collection objects associated with the collecting events
      step_4 = step_3.map(&:collection_objects).flatten.map(&:id).uniq
      retval = CollectionObject.where(id: step_4.sort)
    else
      retval = CollectionObject.joins(:geographic_items)
        .where(GeographicItem.contained_by_where_sql(geographic_item.id))
        .limit(limit)
        .includes(:data_attributes, :collecting_event)
    end
    retval
  end

  # TODO: deprecate
  def self.selected_column_names
    @selected_column_names = {
      ce: {in: {}, im: {}},
      co: {in: {}, im: {}},
      bc: {in: {}, im: {}}
    } if @selected_column_names.nil?
    @selected_column_names
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collecting events
  # decode which headers to be displayed for collecting events
  def self.ce_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:ce][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:ce][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.ce_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.collecting_event.internal_attributes
      all_import_das   = collection_object.collecting_event.import_attributes
      group            = collection[:ce]
      unless group.nil?
        group.each_key { |type_key|
          group[type_key.to_sym].each_key { |header|
            this_val = nil
            case type_key.to_sym
            when :in
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            when :im
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            else
            end
          }
        }
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collection objects
  # decode which headers to be displayed for collection objects
  def self.co_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:co][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:co][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.co_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.internal_attributes
      all_import_das   = collection_object.import_attributes
      group            = collection[:co]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = nil
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
        unless group.empty?
          unless group[:im].empty?
            group[:im].each_key { |header|
              this_val = nil
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for biocuration classifications
  # decode which headers to be displayed for biocuration classifications
  def self.bc_headers(project_id)
    CollectionObject.selected_column_names
    # add selectable column names (unselected) to the column name list list
    BiocurationClass.where(project_id:).map(&:name).each { |column_name|
      @selected_column_names[:bc][:in][column_name] = {checked: '0'}
    }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.bc_attributes(collection_object, col_defs)
    retval = []
    collection = col_defs
    unless collection.nil?
      group = collection[:bc]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = collection_object.biocuration_classes.map(&:name).include?(header) ? '1' : '0'
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Array] collecting_event_ids (e.g., from CollectingEvent.in_date_range)
  # @param [Array] area_object_ids (e.g., from GeographicItem.gather_selected_data())
  # @return [Scope] of intersection of collecting events (usually by date range)
  #   and collection objects (usually by inclusion in geographic areas/items)
  def self.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id)
    collecting_events_clause = {collecting_event_id: collecting_event_ids, project: project_id}
    area_objects_clause      = {id: area_object_ids, project: project_id}

    if (collecting_event_ids.empty?)
      collecting_events_clause = {project: project_id}
    end

    if (area_object_ids.empty?)
      area_objects_clause = {}
      if (area_set)
        area_objects_clause = 'false'
      end
    end

    retval = CollectionObject.joins(:collecting_event)
      .where(collecting_events_clause)
      .where(area_objects_clause)
    retval
  end

  # TODO: move to filter
  # @param [Hash] search_start_date string in form 'yyyy-mm-dd'
  # @param [Hash] search_end_date string in form 'yyyy-mm-dd'
  # @param [Hash] partial_overlap 'on' or 'off'
  # @return [Scope] of selected collection objects through collecting events with georeferences, remember to scope to project!
  def self.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on')
    allow_partial = (partial_overlap.downcase == 'off' ? false : true) # TODO: Just get the correct values from the form!
    q = Queries::CollectingEvent::Filter.new(start_date: search_start_date, end_date: search_end_date, partial_overlap_dates: allow_partial)
    joins(:collecting_event).where(q.between_date_range_facet.to_sql)
  end

  # @param used_on [String] required, one of `TaxonDetermination`, `BiologicalAssociation`
  # @return [Scope]
  #    the max 10 most recently used collection_objects, as `used_on`
  def self.used_recently(user_id, project_id, used_on = '')
    return [] if used_on != 'TaxonDetermination' && used_on != 'BiologicalAssociation'
    t = case used_on
        when 'TaxonDetermination'
          TaxonDetermination.arel_table
        when 'BiologicalAssociation'
          BiologicalAssociation.arel_table
        end

    p = CollectionObject.arel_table

    # i is a select manager
    i = case used_on
        when 'BiologicalAssociation'
          t.project(t['biological_association_subject_id'], t['updated_at']).from(t)
            .where(
              t['updated_at'].gt(1.week.ago).and(
                t['biological_association_subject_type'].eq('CollectionObject')
              )
            )
              .where(t['updated_by_id'].eq(user_id))
              .where(t['project_id'].eq(project_id))
              .order(t['updated_at'].desc)
        else
          # TODO: update to reference new TaxonDetermination
          t.project(t['taxon_determination_object_id'], t['taxon_determination_object_type'], t['updated_at']).from(t)
            .where(t['updated_at'].gt( 1.week.ago ))
            .where(t['updated_by_id'].eq(user_id))
            .where(t['project_id'].eq(project_id))
            .order(t['updated_at'].desc)
        end

    # z is a table alias
    z = i.as('recent_t')

    j = case used_on
        when 'BiologicalAssociation'
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(
            z['biological_association_subject_id'].eq(p['id'])
          ))
        else
          # TODO: needs to be fixed to scope the taxon_determination_object_type
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(z['taxon_determination_object_id'].eq(p['id'])))
        end

    CollectionObject.joins(j).pluck(:id).uniq
  end

  # @params target [String] one of `TaxonDetermination`, `BiologicalAssociation` , nil
  # @return [Hash] otus optimized for user selection
  def self.select_optimized(user_id, project_id, target = nil)
    r = used_recently(user_id, project_id, target)
    h = {
      quick: [],
      pinboard: CollectionObject.pinned_by(user_id).where(project_id:).to_a,
      recent: []
    }

    if target && !r.empty?
      n = target.tableize.to_sym
      h[:recent] = CollectionObject.where('"collection_objects"."id" IN (?)', r.first(10) ).to_a
      h[:quick] = (CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a  +
                   CollectionObject.where('"collection_objects"."id" IN (?)', r.first(4) ).to_a).uniq
    else
      h[:recent] = CollectionObject.where(project_id:, updated_by_id: user_id).order('updated_at DESC').limit(10).to_a
      h[:quick] = CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a
    end

    h
  end

  # TODO: Unify with Extract in concern
  # @return [Identifier::Local::CatalogNumber, nil]
  #   the first (position) catalog number for this collection object, either on specimen, or container
  def preferred_catalog_number
    if i = Identifier::Local::CatalogNumber.where(identifier_object: self).order(:position).first
      i
    else
      if container
        container.identifiers.where(identifiers: {type: 'Identifier::Local::CatalogNumber'}).order(:position).first
      else
        nil
      end
    end
  end

  # @return [Identifier::Local::RecordNumber, nil]
  #   the first (position) record_Number, on a specimen
  #   !1 Doesn't presently support containers
  def preferred_record_number
    Identifier::Local::RecordNumber.where(identifier_object: self).order(:position).first
  end

  def geographic_name_classification
    # don't load the whole object, just the fields we need
    if a = DwcOccurrence.where(dwc_occurrence_object: self).select(:country, :stateProvince, :county).first

      c = a.country
      s = a.stateProvince
      y = a.county

      v = ::Utilities::Geo::DICTIONARY[c]
      c = v if v
      # s = v if v = ::Utilities::Geo::DICTIONARY[s] # None in there yet
      # y = v if v = ::Utilities::Geo::DICTIONARY[y] # None in there yet

      return {
        country: c,
        state: s,
        county: y
      }
    end
  end

  # return [Boolean]
  #    True if instance is a subclass of BiologicalCollectionObject
  def is_biological?
    self.class <= BiologicalCollectionObject ? true : false
  end

  def annotations
    h = annotations_hash
    (h['biocuration classifications'] = biocuration_classes) if is_biological? && biocuration_classifications.load.any?
    h
  end

  def sv_missing_accession_fields
    soft_validations.add(:accessioned_at, 'Date is not selected') if self.accessioned_at.nil? && !self.accession_provider.nil?
    soft_validations.add(:base, 'Provider is not selected') if !self.accessioned_at.nil? && self.accession_provider.nil?
  end

  def sv_missing_deaccession_fields
    soft_validations.add(:deaccessioned_at, 'Date is not selected') if self.deaccessioned_at.nil? && self.deaccession_reason.present?
    soft_validations.add(:base, 'Recipient is not selected') if self.deaccession_recipient.nil? && self.deaccession_reason && self.deaccessioned_at
    soft_validations.add(:deaccession_reason, 'Reason is is not defined') if self.deaccession_reason.blank? && self.deaccession_recipient && self.deaccessioned_at
  end

  def sv_missing_determination
    # see biological_collection_object
  end

  def sv_missing_collecting_event
    # see biological_collection_object
  end

  def sv_missing_preparation_type
    # see biological_collection_object
  end

  def sv_missing_repository
    # WHY? -  see biological_collection_object
  end

  def sv_missing_biocuration_classification
    # see biological_collection_object
  end

  # See Depiction#destroy_image_stub_collection_object
  # Used to determin if the CO can be
  # destroy after moving an image off
  # this object.
  def is_image_stub?
    r = [
      collecting_event_id.blank?,
      !depictions.reload.any?,
      identifiers.count <= 1,
      !taxon_determinations.any?,
      !type_materials.any?,
      !citations.any?,
      !data_attributes.any?,
      !notes.any?,
      !observations.any?
    ]

   !r.include?(false)

  end

  protected

  def collecting_event_belongs_to_project
    if collecting_event&.persisted? && (Current.project_id || project_id)
      errors.add(:base, 'collecting event is not from this project') if collecting_event.project_id != (Current.project_id || project_id)
    end
  end

  def check_that_both_of_category_and_total_are_not_present
    errors.add(:ranged_lot_category_id, 'Both ranged_lot_category and total can not be set') if ranged_lot_category_id.present? && total.present?
  end

  def check_that_either_total_or_ranged_lot_category_id_is_present
    errors.add(:base, 'Either total or a ranged lot category must be provided') if ranged_lot_category_id.blank? && total.blank?
  end

  def assign_type_if_total_or_ranged_lot_category_id_provided
    if self.total == 1
      self.type = 'Specimen'
    elsif self.total.to_i > 1
      self.type = 'Lot'
    elsif total.nil? && ranged_lot_category_id.present?
      self.type = 'RangedLot'
    end
    true
  end

  def reject_collecting_event(attributed)
    reject = true
    CollectingEvent.core_attributes.each do |a|
      if attributed[a].present?
        reject = false
        break
      end
    end
    # !! does not account for georeferences_attributes!
    reject
  end

end

#buffered_other_labels ⇒ String

An incoming, typically verbatim, block of data, as typically found on label that is unrelated to determinations or collecting events. All buffered_ attributes are written but not intended to be deleted or otherwise updated. Buffered_ attributes are typically only used in rapid data capture, primarily in historical situations.

Returns:

• (String)

# File 'app/models/collection_object.rb', line 64

class CollectionObject < ApplicationRecord
  include GlobalID::Identification
  include Housekeeping

  include Shared::Citations
  include Shared::Containable
  include Shared::DataAttributes
  include Shared::Loanable
  include Shared::Identifiers
  include Shared::Notes
  include Shared::Tags
  include Shared::Depictions
  include Shared::OriginRelationship
  include Shared::Confidences
  include Shared::ProtocolRelationships
  include Shared::HasPapertrail
  include Shared::Observations
  include Shared::IsData
  include Shared::QueryBatchUpdate
  include SoftValidation

  include Shared::BiologicalExtensions

  include Shared::Taxonomy # at present must be before IsDwcOccurence
  include Shared::IsDwcOccurrence
  include CollectionObject::DwcExtensions

  ignore_whitespace_on(:buffered_collecting_event, :buffered_determinations, :buffered_other_labels)

  # TODO: move to export
  CO_OTU_HEADERS = %w{OTU OTU\ name Family Genus Species Country State County Locality Latitude Longitude}.freeze

  BUFFERED_ATTRIBUTES = %i{buffered_collecting_event buffered_determinations buffered_other_labels}.freeze

  GRAPH_ENTRY_POINTS = [:biological_associations, :data_attributes, :taxon_determinations, :biocuration_classifications, :collecting_event, :origin_relationships, :extracts, :observation_matrices]

  # Identifier delegations
  # .catalog_number_cached
  delegate :cached, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true
  # .catalog_number_namespace
  delegate :namespace, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true

  # .record_number_cached
  delegate :cached, to: :preferred_record_number, prefix: :record_number, allow_nil: true
  # .record_number_namespace
  delegate :namespace, to: :preferred_record_number, prefix: :record_number, allow_nil: true

  # CollectingEvent delegations
  delegate :map_center, to: :collecting_event, prefix: :collecting_event, allow_nil: true
  delegate :collectors, to: :collecting_event, prefix: :collecting_event, allow_nil: true

  # Repository delegations
  delegate :acronym, to: :repository, prefix: :repository, allow_nil: true
  delegate :url, to: :repository, prefix: :repository, allow_nil: true
  delegate :institutional_LSID, to: :repository, prefix: :repository, allow_nil: true

  # Preparation delegations
  delegate :name, to: :preparation_type, prefix: :preparation_type, allow_nil: true

  has_one :accession_provider_role, class_name: 'AccessionProvider', as: :role_object, dependent: :destroy
  has_one :accession_provider, through: :accession_provider_role, source: :person
  has_one :deaccession_recipient_role, class_name: 'DeaccessionRecipient', as: :role_object, dependent: :destroy
  has_one :deaccession_recipient, through: :deaccession_recipient_role, source: :person

  # TODO: Deprecate these models.  Semantics also confuse with origin relationship.
  has_many :derived_collection_objects, inverse_of: :collection_object, dependent: :restrict_with_error
  has_many :collection_object_observations, through: :derived_collection_objects, inverse_of: :collection_objects

  has_many :sqed_depictions, through: :depictions, dependent: :restrict_with_error

  belongs_to :collecting_event, inverse_of: :collection_objects
  belongs_to :preparation_type, inverse_of: :collection_objects
  belongs_to :ranged_lot_category, inverse_of: :ranged_lots
  belongs_to :repository, inverse_of: :collection_objects
  belongs_to :current_repository, class_name: 'Repository', inverse_of: :collection_objects

  has_many :georeferences, through: :collecting_event
  has_many :geographic_items, through: :georeferences

  has_many :collectors, through: :collecting_event

  accepts_nested_attributes_for :collecting_event, allow_destroy: true, reject_if: :reject_collecting_event

  before_validation :assign_type_if_total_or_ranged_lot_category_id_provided

  validates_presence_of :type
  validate :check_that_either_total_or_ranged_lot_category_id_is_present
  validate :check_that_both_of_category_and_total_are_not_present
  validate :collecting_event_belongs_to_project

  soft_validate(
    :sv_missing_accession_fields,
    set: :missing_accession_fields,
    name: 'Missing accession fields',
    description: 'Name or Provider are not selected')

  soft_validate(
    :sv_missing_deaccession_fields,
    set: :missing_deaccession_fields,
    name: 'Missing deaccesson fields',
    description: 'Date, recipient, or reason are not specified')

  scope :with_sequence_name, ->(name) { joins(sequence_join_hack_sql).where(sequences: {name:}) }
  scope :via_descriptor, ->(descriptor) { joins(sequence_join_hack_sql).where(sequences: {id: descriptor.sequences}) }

  has_many :extracts, through: :origin_relationships, source: :new_object, source_type: 'Extract'
  has_many :sequences, through: :extracts

  # This is a hack, maybe related to a Rails 5.1 bug.
  # It returns the SQL that works in 5.0/4.2 that
  # links CollectionObject to Sequences:
  # joins(derived_extracts: [:derived_sequences])
  def self.sequence_join_hack_sql
    %Q{INNER JOIN  "origin_relationships"
               ON  "origin_relationships"."old_object_id" = "collection_objects"."id"
                  AND  "origin_relationships"."new_object_type" = 'Extract'
                  AND  "origin_relationships"."old_object_type" = 'CollectionObject'
       INNER JOIN  "extracts"
               ON  "extracts"."id" =  "origin_relationships"."new_object_id"
       INNER JOIN  "origin_relationships" "origin_relationships_extracts_join"
               ON  "origin_relationships_extracts_join"."old_object_id" = "extracts"."id"
                  AND  "origin_relationships_extracts_join"."new_object_type" = 'Sequence'
                  AND  "origin_relationships_extracts_join"."old_object_type" = 'Extract'
       INNER JOIN  "sequences"
               ON  "sequences"."id" = "origin_relationships_extracts_join"."new_object_id"}
  end

  def self.batch_update(params)
    request = QueryBatchRequest.new(
      async_cutoff: params[:async_cutoff] || 50,
      klass: 'CollectionObject',
      object_filter_params: params[:collection_object_query],
      object_params: params[:collection_object],
      preview: params[:preview],
    )

    request.cap = 1000

    query_batch_update(request)
  end

  def self.batch_update_dwc_occurrence(params)
    q = Queries::CollectionObject::Filter.new(params).all

    r = BatchResponse.new
    r.method = 'batch_update_dwc_occurrence'
    r.klass = 'CollectionObject'

    c = q.all.count

    if c == 0 || c > 10000
      r.cap_reason = 'Too many (or no) collection objects (max 10k)'
      return r
    end

    if c < 51
      q.each do |co|
        co.set_dwc_occurrence
        r.updated.push co.id
      end
    else
      r.async = true
      q.each do |co|
        co.dwc_occurrence_update_query
      end
    end

    return r
  end

  def dwc_occurrence_update_query
    self.send(:set_dwc_occurrence)
  end

  handle_asynchronously :dwc_occurrence_update_query, run_at: Proc.new { 1.second.from_now }, queue: :query_batch_update

  # TODO: move to a helper
  def self.breakdown_status(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array

    breakdown = {
      total_objects:     collection_objects.length,
      collecting_events: {},
      determinations:    {},
      bio_overview:      []
    }

    breakdown.merge!(breakdown_buffered(collection_objects))

    collection_objects.each do |co|
      breakdown[:collecting_events].merge!(co => co.collecting_event) if co.collecting_event
      breakdown[:determinations].merge!(co => co.taxon_determinations) if co.taxon_determinations.load.any?
      breakdown[:bio_overview].push([co.total, co.biocuration_classes.collect { |a| a.name }])
    end

    breakdown
  end

  # @return [Hash]
  #   a unque list of buffered_ values observed in the collection objects passed
  def self.breakdown_buffered(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array
    breakdown = {}
    categories = BUFFERED_ATTRIBUTES

    categories.each do |c|
      breakdown[c] = []
    end

    categories.each do |c|
      collection_objects.each do |co|
        breakdown[c].push co.send(c)
      end
    end

    categories.each do |c|
      breakdown[c].uniq!
    end

    breakdown
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.earliest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:end_date_year)

    return EARLIEST_DATE if a.nil? && b.nil?  # 1700-01-01

    d = nil

    if a && b
      if a < b
        d = a
      end
    else
      d = a || b
    end
    d.to_s + '-01-01'
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.latest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:end_date_year)

    c = Time.now.strftime('%Y-%m-%d')

    return c if a.nil? && b.nil?

    d = nil

    if a && b
      if a > b
        d = a
      end
    else
      d = a || b
    end

    d.to_s + '/12/31'
  end

  # TODO: Clarify this.
  # CAREFULL - this isn't _in_, this is *with*, if it was in it would be spatial query, not a join(:geographic_items)
  #
  # Find all collection objects which have collecting events which have georeferences which have geographic_items which
  # are located within the geographic item supplied
  # @param [GeographicItem] geographic_item_id
  # @return [Scope] of CollectionObject
  def self.in_geographic_item(geographic_item, limit, steps = false)
    geographic_item_id = geographic_item.id
    if steps
      gi = GeographicItem.find(geographic_item_id)
      # find the geographic_items inside gi
      step_1 = GeographicItem.is_contained_by('any', gi) # .pluck(:id)
      # find the georeferences from the geographic_items
      step_2 = step_1.map(&:georeferences).uniq.flatten
      # find the collecting events connected to the georeferences
      step_3 = step_2.map(&:collecting_event).uniq.flatten
      # find the collection objects associated with the collecting events
      step_4 = step_3.map(&:collection_objects).flatten.map(&:id).uniq
      retval = CollectionObject.where(id: step_4.sort)
    else
      retval = CollectionObject.joins(:geographic_items)
        .where(GeographicItem.contained_by_where_sql(geographic_item.id))
        .limit(limit)
        .includes(:data_attributes, :collecting_event)
    end
    retval
  end

  # TODO: deprecate
  def self.selected_column_names
    @selected_column_names = {
      ce: {in: {}, im: {}},
      co: {in: {}, im: {}},
      bc: {in: {}, im: {}}
    } if @selected_column_names.nil?
    @selected_column_names
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collecting events
  # decode which headers to be displayed for collecting events
  def self.ce_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:ce][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:ce][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.ce_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.collecting_event.internal_attributes
      all_import_das   = collection_object.collecting_event.import_attributes
      group            = collection[:ce]
      unless group.nil?
        group.each_key { |type_key|
          group[type_key.to_sym].each_key { |header|
            this_val = nil
            case type_key.to_sym
            when :in
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            when :im
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            else
            end
          }
        }
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collection objects
  # decode which headers to be displayed for collection objects
  def self.co_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:co][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:co][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.co_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.internal_attributes
      all_import_das   = collection_object.import_attributes
      group            = collection[:co]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = nil
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
        unless group.empty?
          unless group[:im].empty?
            group[:im].each_key { |header|
              this_val = nil
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for biocuration classifications
  # decode which headers to be displayed for biocuration classifications
  def self.bc_headers(project_id)
    CollectionObject.selected_column_names
    # add selectable column names (unselected) to the column name list list
    BiocurationClass.where(project_id:).map(&:name).each { |column_name|
      @selected_column_names[:bc][:in][column_name] = {checked: '0'}
    }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.bc_attributes(collection_object, col_defs)
    retval = []
    collection = col_defs
    unless collection.nil?
      group = collection[:bc]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = collection_object.biocuration_classes.map(&:name).include?(header) ? '1' : '0'
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Array] collecting_event_ids (e.g., from CollectingEvent.in_date_range)
  # @param [Array] area_object_ids (e.g., from GeographicItem.gather_selected_data())
  # @return [Scope] of intersection of collecting events (usually by date range)
  #   and collection objects (usually by inclusion in geographic areas/items)
  def self.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id)
    collecting_events_clause = {collecting_event_id: collecting_event_ids, project: project_id}
    area_objects_clause      = {id: area_object_ids, project: project_id}

    if (collecting_event_ids.empty?)
      collecting_events_clause = {project: project_id}
    end

    if (area_object_ids.empty?)
      area_objects_clause = {}
      if (area_set)
        area_objects_clause = 'false'
      end
    end

    retval = CollectionObject.joins(:collecting_event)
      .where(collecting_events_clause)
      .where(area_objects_clause)
    retval
  end

  # TODO: move to filter
  # @param [Hash] search_start_date string in form 'yyyy-mm-dd'
  # @param [Hash] search_end_date string in form 'yyyy-mm-dd'
  # @param [Hash] partial_overlap 'on' or 'off'
  # @return [Scope] of selected collection objects through collecting events with georeferences, remember to scope to project!
  def self.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on')
    allow_partial = (partial_overlap.downcase == 'off' ? false : true) # TODO: Just get the correct values from the form!
    q = Queries::CollectingEvent::Filter.new(start_date: search_start_date, end_date: search_end_date, partial_overlap_dates: allow_partial)
    joins(:collecting_event).where(q.between_date_range_facet.to_sql)
  end

  # @param used_on [String] required, one of `TaxonDetermination`, `BiologicalAssociation`
  # @return [Scope]
  #    the max 10 most recently used collection_objects, as `used_on`
  def self.used_recently(user_id, project_id, used_on = '')
    return [] if used_on != 'TaxonDetermination' && used_on != 'BiologicalAssociation'
    t = case used_on
        when 'TaxonDetermination'
          TaxonDetermination.arel_table
        when 'BiologicalAssociation'
          BiologicalAssociation.arel_table
        end

    p = CollectionObject.arel_table

    # i is a select manager
    i = case used_on
        when 'BiologicalAssociation'
          t.project(t['biological_association_subject_id'], t['updated_at']).from(t)
            .where(
              t['updated_at'].gt(1.week.ago).and(
                t['biological_association_subject_type'].eq('CollectionObject')
              )
            )
              .where(t['updated_by_id'].eq(user_id))
              .where(t['project_id'].eq(project_id))
              .order(t['updated_at'].desc)
        else
          # TODO: update to reference new TaxonDetermination
          t.project(t['taxon_determination_object_id'], t['taxon_determination_object_type'], t['updated_at']).from(t)
            .where(t['updated_at'].gt( 1.week.ago ))
            .where(t['updated_by_id'].eq(user_id))
            .where(t['project_id'].eq(project_id))
            .order(t['updated_at'].desc)
        end

    # z is a table alias
    z = i.as('recent_t')

    j = case used_on
        when 'BiologicalAssociation'
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(
            z['biological_association_subject_id'].eq(p['id'])
          ))
        else
          # TODO: needs to be fixed to scope the taxon_determination_object_type
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(z['taxon_determination_object_id'].eq(p['id'])))
        end

    CollectionObject.joins(j).pluck(:id).uniq
  end

  # @params target [String] one of `TaxonDetermination`, `BiologicalAssociation` , nil
  # @return [Hash] otus optimized for user selection
  def self.select_optimized(user_id, project_id, target = nil)
    r = used_recently(user_id, project_id, target)
    h = {
      quick: [],
      pinboard: CollectionObject.pinned_by(user_id).where(project_id:).to_a,
      recent: []
    }

    if target && !r.empty?
      n = target.tableize.to_sym
      h[:recent] = CollectionObject.where('"collection_objects"."id" IN (?)', r.first(10) ).to_a
      h[:quick] = (CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a  +
                   CollectionObject.where('"collection_objects"."id" IN (?)', r.first(4) ).to_a).uniq
    else
      h[:recent] = CollectionObject.where(project_id:, updated_by_id: user_id).order('updated_at DESC').limit(10).to_a
      h[:quick] = CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a
    end

    h
  end

  # TODO: Unify with Extract in concern
  # @return [Identifier::Local::CatalogNumber, nil]
  #   the first (position) catalog number for this collection object, either on specimen, or container
  def preferred_catalog_number
    if i = Identifier::Local::CatalogNumber.where(identifier_object: self).order(:position).first
      i
    else
      if container
        container.identifiers.where(identifiers: {type: 'Identifier::Local::CatalogNumber'}).order(:position).first
      else
        nil
      end
    end
  end

  # @return [Identifier::Local::RecordNumber, nil]
  #   the first (position) record_Number, on a specimen
  #   !1 Doesn't presently support containers
  def preferred_record_number
    Identifier::Local::RecordNumber.where(identifier_object: self).order(:position).first
  end

  def geographic_name_classification
    # don't load the whole object, just the fields we need
    if a = DwcOccurrence.where(dwc_occurrence_object: self).select(:country, :stateProvince, :county).first

      c = a.country
      s = a.stateProvince
      y = a.county

      v = ::Utilities::Geo::DICTIONARY[c]
      c = v if v
      # s = v if v = ::Utilities::Geo::DICTIONARY[s] # None in there yet
      # y = v if v = ::Utilities::Geo::DICTIONARY[y] # None in there yet

      return {
        country: c,
        state: s,
        county: y
      }
    end
  end

  # return [Boolean]
  #    True if instance is a subclass of BiologicalCollectionObject
  def is_biological?
    self.class <= BiologicalCollectionObject ? true : false
  end

  def annotations
    h = annotations_hash
    (h['biocuration classifications'] = biocuration_classes) if is_biological? && biocuration_classifications.load.any?
    h
  end

  def sv_missing_accession_fields
    soft_validations.add(:accessioned_at, 'Date is not selected') if self.accessioned_at.nil? && !self.accession_provider.nil?
    soft_validations.add(:base, 'Provider is not selected') if !self.accessioned_at.nil? && self.accession_provider.nil?
  end

  def sv_missing_deaccession_fields
    soft_validations.add(:deaccessioned_at, 'Date is not selected') if self.deaccessioned_at.nil? && self.deaccession_reason.present?
    soft_validations.add(:base, 'Recipient is not selected') if self.deaccession_recipient.nil? && self.deaccession_reason && self.deaccessioned_at
    soft_validations.add(:deaccession_reason, 'Reason is is not defined') if self.deaccession_reason.blank? && self.deaccession_recipient && self.deaccessioned_at
  end

  def sv_missing_determination
    # see biological_collection_object
  end

  def sv_missing_collecting_event
    # see biological_collection_object
  end

  def sv_missing_preparation_type
    # see biological_collection_object
  end

  def sv_missing_repository
    # WHY? -  see biological_collection_object
  end

  def sv_missing_biocuration_classification
    # see biological_collection_object
  end

  # See Depiction#destroy_image_stub_collection_object
  # Used to determin if the CO can be
  # destroy after moving an image off
  # this object.
  def is_image_stub?
    r = [
      collecting_event_id.blank?,
      !depictions.reload.any?,
      identifiers.count <= 1,
      !taxon_determinations.any?,
      !type_materials.any?,
      !citations.any?,
      !data_attributes.any?,
      !notes.any?,
      !observations.any?
    ]

   !r.include?(false)

  end

  protected

  def collecting_event_belongs_to_project
    if collecting_event&.persisted? && (Current.project_id || project_id)
      errors.add(:base, 'collecting event is not from this project') if collecting_event.project_id != (Current.project_id || project_id)
    end
  end

  def check_that_both_of_category_and_total_are_not_present
    errors.add(:ranged_lot_category_id, 'Both ranged_lot_category and total can not be set') if ranged_lot_category_id.present? && total.present?
  end

  def check_that_either_total_or_ranged_lot_category_id_is_present
    errors.add(:base, 'Either total or a ranged lot category must be provided') if ranged_lot_category_id.blank? && total.blank?
  end

  def assign_type_if_total_or_ranged_lot_category_id_provided
    if self.total == 1
      self.type = 'Specimen'
    elsif self.total.to_i > 1
      self.type = 'Lot'
    elsif total.nil? && ranged_lot_category_id.present?
      self.type = 'RangedLot'
    end
    true
  end

  def reject_collecting_event(attributed)
    reject = true
    CollectingEvent.core_attributes.each do |a|
      if attributed[a].present?
        reject = false
        break
      end
    end
    # !! does not account for georeferences_attributes!
    reject
  end

end

#collecting_event_id ⇒ Integer

The id of the collecting event from whence this object came. See CollectingEvent.

Returns:

• (Integer)

# File 'app/models/collection_object.rb', line 64

class CollectionObject < ApplicationRecord
  include GlobalID::Identification
  include Housekeeping

  include Shared::Citations
  include Shared::Containable
  include Shared::DataAttributes
  include Shared::Loanable
  include Shared::Identifiers
  include Shared::Notes
  include Shared::Tags
  include Shared::Depictions
  include Shared::OriginRelationship
  include Shared::Confidences
  include Shared::ProtocolRelationships
  include Shared::HasPapertrail
  include Shared::Observations
  include Shared::IsData
  include Shared::QueryBatchUpdate
  include SoftValidation

  include Shared::BiologicalExtensions

  include Shared::Taxonomy # at present must be before IsDwcOccurence
  include Shared::IsDwcOccurrence
  include CollectionObject::DwcExtensions

  ignore_whitespace_on(:buffered_collecting_event, :buffered_determinations, :buffered_other_labels)

  # TODO: move to export
  CO_OTU_HEADERS = %w{OTU OTU\ name Family Genus Species Country State County Locality Latitude Longitude}.freeze

  BUFFERED_ATTRIBUTES = %i{buffered_collecting_event buffered_determinations buffered_other_labels}.freeze

  GRAPH_ENTRY_POINTS = [:biological_associations, :data_attributes, :taxon_determinations, :biocuration_classifications, :collecting_event, :origin_relationships, :extracts, :observation_matrices]

  # Identifier delegations
  # .catalog_number_cached
  delegate :cached, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true
  # .catalog_number_namespace
  delegate :namespace, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true

  # .record_number_cached
  delegate :cached, to: :preferred_record_number, prefix: :record_number, allow_nil: true
  # .record_number_namespace
  delegate :namespace, to: :preferred_record_number, prefix: :record_number, allow_nil: true

  # CollectingEvent delegations
  delegate :map_center, to: :collecting_event, prefix: :collecting_event, allow_nil: true
  delegate :collectors, to: :collecting_event, prefix: :collecting_event, allow_nil: true

  # Repository delegations
  delegate :acronym, to: :repository, prefix: :repository, allow_nil: true
  delegate :url, to: :repository, prefix: :repository, allow_nil: true
  delegate :institutional_LSID, to: :repository, prefix: :repository, allow_nil: true

  # Preparation delegations
  delegate :name, to: :preparation_type, prefix: :preparation_type, allow_nil: true

  has_one :accession_provider_role, class_name: 'AccessionProvider', as: :role_object, dependent: :destroy
  has_one :accession_provider, through: :accession_provider_role, source: :person
  has_one :deaccession_recipient_role, class_name: 'DeaccessionRecipient', as: :role_object, dependent: :destroy
  has_one :deaccession_recipient, through: :deaccession_recipient_role, source: :person

  # TODO: Deprecate these models.  Semantics also confuse with origin relationship.
  has_many :derived_collection_objects, inverse_of: :collection_object, dependent: :restrict_with_error
  has_many :collection_object_observations, through: :derived_collection_objects, inverse_of: :collection_objects

  has_many :sqed_depictions, through: :depictions, dependent: :restrict_with_error

  belongs_to :collecting_event, inverse_of: :collection_objects
  belongs_to :preparation_type, inverse_of: :collection_objects
  belongs_to :ranged_lot_category, inverse_of: :ranged_lots
  belongs_to :repository, inverse_of: :collection_objects
  belongs_to :current_repository, class_name: 'Repository', inverse_of: :collection_objects

  has_many :georeferences, through: :collecting_event
  has_many :geographic_items, through: :georeferences

  has_many :collectors, through: :collecting_event

  accepts_nested_attributes_for :collecting_event, allow_destroy: true, reject_if: :reject_collecting_event

  before_validation :assign_type_if_total_or_ranged_lot_category_id_provided

  validates_presence_of :type
  validate :check_that_either_total_or_ranged_lot_category_id_is_present
  validate :check_that_both_of_category_and_total_are_not_present
  validate :collecting_event_belongs_to_project

  soft_validate(
    :sv_missing_accession_fields,
    set: :missing_accession_fields,
    name: 'Missing accession fields',
    description: 'Name or Provider are not selected')

  soft_validate(
    :sv_missing_deaccession_fields,
    set: :missing_deaccession_fields,
    name: 'Missing deaccesson fields',
    description: 'Date, recipient, or reason are not specified')

  scope :with_sequence_name, ->(name) { joins(sequence_join_hack_sql).where(sequences: {name:}) }
  scope :via_descriptor, ->(descriptor) { joins(sequence_join_hack_sql).where(sequences: {id: descriptor.sequences}) }

  has_many :extracts, through: :origin_relationships, source: :new_object, source_type: 'Extract'
  has_many :sequences, through: :extracts

  # This is a hack, maybe related to a Rails 5.1 bug.
  # It returns the SQL that works in 5.0/4.2 that
  # links CollectionObject to Sequences:
  # joins(derived_extracts: [:derived_sequences])
  def self.sequence_join_hack_sql
    %Q{INNER JOIN  "origin_relationships"
               ON  "origin_relationships"."old_object_id" = "collection_objects"."id"
                  AND  "origin_relationships"."new_object_type" = 'Extract'
                  AND  "origin_relationships"."old_object_type" = 'CollectionObject'
       INNER JOIN  "extracts"
               ON  "extracts"."id" =  "origin_relationships"."new_object_id"
       INNER JOIN  "origin_relationships" "origin_relationships_extracts_join"
               ON  "origin_relationships_extracts_join"."old_object_id" = "extracts"."id"
                  AND  "origin_relationships_extracts_join"."new_object_type" = 'Sequence'
                  AND  "origin_relationships_extracts_join"."old_object_type" = 'Extract'
       INNER JOIN  "sequences"
               ON  "sequences"."id" = "origin_relationships_extracts_join"."new_object_id"}
  end

  def self.batch_update(params)
    request = QueryBatchRequest.new(
      async_cutoff: params[:async_cutoff] || 50,
      klass: 'CollectionObject',
      object_filter_params: params[:collection_object_query],
      object_params: params[:collection_object],
      preview: params[:preview],
    )

    request.cap = 1000

    query_batch_update(request)
  end

  def self.batch_update_dwc_occurrence(params)
    q = Queries::CollectionObject::Filter.new(params).all

    r = BatchResponse.new
    r.method = 'batch_update_dwc_occurrence'
    r.klass = 'CollectionObject'

    c = q.all.count

    if c == 0 || c > 10000
      r.cap_reason = 'Too many (or no) collection objects (max 10k)'
      return r
    end

    if c < 51
      q.each do |co|
        co.set_dwc_occurrence
        r.updated.push co.id
      end
    else
      r.async = true
      q.each do |co|
        co.dwc_occurrence_update_query
      end
    end

    return r
  end

  def dwc_occurrence_update_query
    self.send(:set_dwc_occurrence)
  end

  handle_asynchronously :dwc_occurrence_update_query, run_at: Proc.new { 1.second.from_now }, queue: :query_batch_update

  # TODO: move to a helper
  def self.breakdown_status(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array

    breakdown = {
      total_objects:     collection_objects.length,
      collecting_events: {},
      determinations:    {},
      bio_overview:      []
    }

    breakdown.merge!(breakdown_buffered(collection_objects))

    collection_objects.each do |co|
      breakdown[:collecting_events].merge!(co => co.collecting_event) if co.collecting_event
      breakdown[:determinations].merge!(co => co.taxon_determinations) if co.taxon_determinations.load.any?
      breakdown[:bio_overview].push([co.total, co.biocuration_classes.collect { |a| a.name }])
    end

    breakdown
  end

  # @return [Hash]
  #   a unque list of buffered_ values observed in the collection objects passed
  def self.breakdown_buffered(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array
    breakdown = {}
    categories = BUFFERED_ATTRIBUTES

    categories.each do |c|
      breakdown[c] = []
    end

    categories.each do |c|
      collection_objects.each do |co|
        breakdown[c].push co.send(c)
      end
    end

    categories.each do |c|
      breakdown[c].uniq!
    end

    breakdown
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.earliest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:end_date_year)

    return EARLIEST_DATE if a.nil? && b.nil?  # 1700-01-01

    d = nil

    if a && b
      if a < b
        d = a
      end
    else
      d = a || b
    end
    d.to_s + '-01-01'
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.latest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:end_date_year)

    c = Time.now.strftime('%Y-%m-%d')

    return c if a.nil? && b.nil?

    d = nil

    if a && b
      if a > b
        d = a
      end
    else
      d = a || b
    end

    d.to_s + '/12/31'
  end

  # TODO: Clarify this.
  # CAREFULL - this isn't _in_, this is *with*, if it was in it would be spatial query, not a join(:geographic_items)
  #
  # Find all collection objects which have collecting events which have georeferences which have geographic_items which
  # are located within the geographic item supplied
  # @param [GeographicItem] geographic_item_id
  # @return [Scope] of CollectionObject
  def self.in_geographic_item(geographic_item, limit, steps = false)
    geographic_item_id = geographic_item.id
    if steps
      gi = GeographicItem.find(geographic_item_id)
      # find the geographic_items inside gi
      step_1 = GeographicItem.is_contained_by('any', gi) # .pluck(:id)
      # find the georeferences from the geographic_items
      step_2 = step_1.map(&:georeferences).uniq.flatten
      # find the collecting events connected to the georeferences
      step_3 = step_2.map(&:collecting_event).uniq.flatten
      # find the collection objects associated with the collecting events
      step_4 = step_3.map(&:collection_objects).flatten.map(&:id).uniq
      retval = CollectionObject.where(id: step_4.sort)
    else
      retval = CollectionObject.joins(:geographic_items)
        .where(GeographicItem.contained_by_where_sql(geographic_item.id))
        .limit(limit)
        .includes(:data_attributes, :collecting_event)
    end
    retval
  end

  # TODO: deprecate
  def self.selected_column_names
    @selected_column_names = {
      ce: {in: {}, im: {}},
      co: {in: {}, im: {}},
      bc: {in: {}, im: {}}
    } if @selected_column_names.nil?
    @selected_column_names
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collecting events
  # decode which headers to be displayed for collecting events
  def self.ce_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:ce][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:ce][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.ce_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.collecting_event.internal_attributes
      all_import_das   = collection_object.collecting_event.import_attributes
      group            = collection[:ce]
      unless group.nil?
        group.each_key { |type_key|
          group[type_key.to_sym].each_key { |header|
            this_val = nil
            case type_key.to_sym
            when :in
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            when :im
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            else
            end
          }
        }
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collection objects
  # decode which headers to be displayed for collection objects
  def self.co_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:co][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:co][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.co_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.internal_attributes
      all_import_das   = collection_object.import_attributes
      group            = collection[:co]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = nil
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
        unless group.empty?
          unless group[:im].empty?
            group[:im].each_key { |header|
              this_val = nil
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for biocuration classifications
  # decode which headers to be displayed for biocuration classifications
  def self.bc_headers(project_id)
    CollectionObject.selected_column_names
    # add selectable column names (unselected) to the column name list list
    BiocurationClass.where(project_id:).map(&:name).each { |column_name|
      @selected_column_names[:bc][:in][column_name] = {checked: '0'}
    }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.bc_attributes(collection_object, col_defs)
    retval = []
    collection = col_defs
    unless collection.nil?
      group = collection[:bc]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = collection_object.biocuration_classes.map(&:name).include?(header) ? '1' : '0'
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Array] collecting_event_ids (e.g., from CollectingEvent.in_date_range)
  # @param [Array] area_object_ids (e.g., from GeographicItem.gather_selected_data())
  # @return [Scope] of intersection of collecting events (usually by date range)
  #   and collection objects (usually by inclusion in geographic areas/items)
  def self.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id)
    collecting_events_clause = {collecting_event_id: collecting_event_ids, project: project_id}
    area_objects_clause      = {id: area_object_ids, project: project_id}

    if (collecting_event_ids.empty?)
      collecting_events_clause = {project: project_id}
    end

    if (area_object_ids.empty?)
      area_objects_clause = {}
      if (area_set)
        area_objects_clause = 'false'
      end
    end

    retval = CollectionObject.joins(:collecting_event)
      .where(collecting_events_clause)
      .where(area_objects_clause)
    retval
  end

  # TODO: move to filter
  # @param [Hash] search_start_date string in form 'yyyy-mm-dd'
  # @param [Hash] search_end_date string in form 'yyyy-mm-dd'
  # @param [Hash] partial_overlap 'on' or 'off'
  # @return [Scope] of selected collection objects through collecting events with georeferences, remember to scope to project!
  def self.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on')
    allow_partial = (partial_overlap.downcase == 'off' ? false : true) # TODO: Just get the correct values from the form!
    q = Queries::CollectingEvent::Filter.new(start_date: search_start_date, end_date: search_end_date, partial_overlap_dates: allow_partial)
    joins(:collecting_event).where(q.between_date_range_facet.to_sql)
  end

  # @param used_on [String] required, one of `TaxonDetermination`, `BiologicalAssociation`
  # @return [Scope]
  #    the max 10 most recently used collection_objects, as `used_on`
  def self.used_recently(user_id, project_id, used_on = '')
    return [] if used_on != 'TaxonDetermination' && used_on != 'BiologicalAssociation'
    t = case used_on
        when 'TaxonDetermination'
          TaxonDetermination.arel_table
        when 'BiologicalAssociation'
          BiologicalAssociation.arel_table
        end

    p = CollectionObject.arel_table

    # i is a select manager
    i = case used_on
        when 'BiologicalAssociation'
          t.project(t['biological_association_subject_id'], t['updated_at']).from(t)
            .where(
              t['updated_at'].gt(1.week.ago).and(
                t['biological_association_subject_type'].eq('CollectionObject')
              )
            )
              .where(t['updated_by_id'].eq(user_id))
              .where(t['project_id'].eq(project_id))
              .order(t['updated_at'].desc)
        else
          # TODO: update to reference new TaxonDetermination
          t.project(t['taxon_determination_object_id'], t['taxon_determination_object_type'], t['updated_at']).from(t)
            .where(t['updated_at'].gt( 1.week.ago ))
            .where(t['updated_by_id'].eq(user_id))
            .where(t['project_id'].eq(project_id))
            .order(t['updated_at'].desc)
        end

    # z is a table alias
    z = i.as('recent_t')

    j = case used_on
        when 'BiologicalAssociation'
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(
            z['biological_association_subject_id'].eq(p['id'])
          ))
        else
          # TODO: needs to be fixed to scope the taxon_determination_object_type
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(z['taxon_determination_object_id'].eq(p['id'])))
        end

    CollectionObject.joins(j).pluck(:id).uniq
  end

  # @params target [String] one of `TaxonDetermination`, `BiologicalAssociation` , nil
  # @return [Hash] otus optimized for user selection
  def self.select_optimized(user_id, project_id, target = nil)
    r = used_recently(user_id, project_id, target)
    h = {
      quick: [],
      pinboard: CollectionObject.pinned_by(user_id).where(project_id:).to_a,
      recent: []
    }

    if target && !r.empty?
      n = target.tableize.to_sym
      h[:recent] = CollectionObject.where('"collection_objects"."id" IN (?)', r.first(10) ).to_a
      h[:quick] = (CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a  +
                   CollectionObject.where('"collection_objects"."id" IN (?)', r.first(4) ).to_a).uniq
    else
      h[:recent] = CollectionObject.where(project_id:, updated_by_id: user_id).order('updated_at DESC').limit(10).to_a
      h[:quick] = CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a
    end

    h
  end

  # TODO: Unify with Extract in concern
  # @return [Identifier::Local::CatalogNumber, nil]
  #   the first (position) catalog number for this collection object, either on specimen, or container
  def preferred_catalog_number
    if i = Identifier::Local::CatalogNumber.where(identifier_object: self).order(:position).first
      i
    else
      if container
        container.identifiers.where(identifiers: {type: 'Identifier::Local::CatalogNumber'}).order(:position).first
      else
        nil
      end
    end
  end

  # @return [Identifier::Local::RecordNumber, nil]
  #   the first (position) record_Number, on a specimen
  #   !1 Doesn't presently support containers
  def preferred_record_number
    Identifier::Local::RecordNumber.where(identifier_object: self).order(:position).first
  end

  def geographic_name_classification
    # don't load the whole object, just the fields we need
    if a = DwcOccurrence.where(dwc_occurrence_object: self).select(:country, :stateProvince, :county).first

      c = a.country
      s = a.stateProvince
      y = a.county

      v = ::Utilities::Geo::DICTIONARY[c]
      c = v if v
      # s = v if v = ::Utilities::Geo::DICTIONARY[s] # None in there yet
      # y = v if v = ::Utilities::Geo::DICTIONARY[y] # None in there yet

      return {
        country: c,
        state: s,
        county: y
      }
    end
  end

  # return [Boolean]
  #    True if instance is a subclass of BiologicalCollectionObject
  def is_biological?
    self.class <= BiologicalCollectionObject ? true : false
  end

  def annotations
    h = annotations_hash
    (h['biocuration classifications'] = biocuration_classes) if is_biological? && biocuration_classifications.load.any?
    h
  end

  def sv_missing_accession_fields
    soft_validations.add(:accessioned_at, 'Date is not selected') if self.accessioned_at.nil? && !self.accession_provider.nil?
    soft_validations.add(:base, 'Provider is not selected') if !self.accessioned_at.nil? && self.accession_provider.nil?
  end

  def sv_missing_deaccession_fields
    soft_validations.add(:deaccessioned_at, 'Date is not selected') if self.deaccessioned_at.nil? && self.deaccession_reason.present?
    soft_validations.add(:base, 'Recipient is not selected') if self.deaccession_recipient.nil? && self.deaccession_reason && self.deaccessioned_at
    soft_validations.add(:deaccession_reason, 'Reason is is not defined') if self.deaccession_reason.blank? && self.deaccession_recipient && self.deaccessioned_at
  end

  def sv_missing_determination
    # see biological_collection_object
  end

  def sv_missing_collecting_event
    # see biological_collection_object
  end

  def sv_missing_preparation_type
    # see biological_collection_object
  end

  def sv_missing_repository
    # WHY? -  see biological_collection_object
  end

  def sv_missing_biocuration_classification
    # see biological_collection_object
  end

  # See Depiction#destroy_image_stub_collection_object
  # Used to determin if the CO can be
  # destroy after moving an image off
  # this object.
  def is_image_stub?
    r = [
      collecting_event_id.blank?,
      !depictions.reload.any?,
      identifiers.count <= 1,
      !taxon_determinations.any?,
      !type_materials.any?,
      !citations.any?,
      !data_attributes.any?,
      !notes.any?,
      !observations.any?
    ]

   !r.include?(false)

  end

  protected

  def collecting_event_belongs_to_project
    if collecting_event&.persisted? && (Current.project_id || project_id)
      errors.add(:base, 'collecting event is not from this project') if collecting_event.project_id != (Current.project_id || project_id)
    end
  end

  def check_that_both_of_category_and_total_are_not_present
    errors.add(:ranged_lot_category_id, 'Both ranged_lot_category and total can not be set') if ranged_lot_category_id.present? && total.present?
  end

  def check_that_either_total_or_ranged_lot_category_id_is_present
    errors.add(:base, 'Either total or a ranged lot category must be provided') if ranged_lot_category_id.blank? && total.blank?
  end

  def assign_type_if_total_or_ranged_lot_category_id_provided
    if self.total == 1
      self.type = 'Specimen'
    elsif self.total.to_i > 1
      self.type = 'Lot'
    elsif total.nil? && ranged_lot_category_id.present?
      self.type = 'RangedLot'
    end
    true
  end

  def reject_collecting_event(attributed)
    reject = true
    CollectingEvent.core_attributes.each do |a|
      if attributed[a].present?
        reject = false
        break
      end
    end
    # !! does not account for georeferences_attributes!
    reject
  end

end

#current_respository_id ⇒ Integer

The id of the current repository. The current repository is the Repository that the specimen can be expected to be found at (i.e. “is localized to”) at the present time. See also respository_id. This is a temporally bound assertion of location of the specimen, not ownership. In the future this will need to be reconciled with concepts of “custody” (the agent responsible for the specimen) and a stricter modelling of localization (in TaxonWorks this really should be a Container::Collection or Container::Building, i.e. the attribute doesn’t really belong here in the long term.

Returns:

• (Integer)

# File 'app/models/collection_object.rb', line 64

class CollectionObject < ApplicationRecord
  include GlobalID::Identification
  include Housekeeping

  include Shared::Citations
  include Shared::Containable
  include Shared::DataAttributes
  include Shared::Loanable
  include Shared::Identifiers
  include Shared::Notes
  include Shared::Tags
  include Shared::Depictions
  include Shared::OriginRelationship
  include Shared::Confidences
  include Shared::ProtocolRelationships
  include Shared::HasPapertrail
  include Shared::Observations
  include Shared::IsData
  include Shared::QueryBatchUpdate
  include SoftValidation

  include Shared::BiologicalExtensions

  include Shared::Taxonomy # at present must be before IsDwcOccurence
  include Shared::IsDwcOccurrence
  include CollectionObject::DwcExtensions

  ignore_whitespace_on(:buffered_collecting_event, :buffered_determinations, :buffered_other_labels)

  # TODO: move to export
  CO_OTU_HEADERS = %w{OTU OTU\ name Family Genus Species Country State County Locality Latitude Longitude}.freeze

  BUFFERED_ATTRIBUTES = %i{buffered_collecting_event buffered_determinations buffered_other_labels}.freeze

  GRAPH_ENTRY_POINTS = [:biological_associations, :data_attributes, :taxon_determinations, :biocuration_classifications, :collecting_event, :origin_relationships, :extracts, :observation_matrices]

  # Identifier delegations
  # .catalog_number_cached
  delegate :cached, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true
  # .catalog_number_namespace
  delegate :namespace, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true

  # .record_number_cached
  delegate :cached, to: :preferred_record_number, prefix: :record_number, allow_nil: true
  # .record_number_namespace
  delegate :namespace, to: :preferred_record_number, prefix: :record_number, allow_nil: true

  # CollectingEvent delegations
  delegate :map_center, to: :collecting_event, prefix: :collecting_event, allow_nil: true
  delegate :collectors, to: :collecting_event, prefix: :collecting_event, allow_nil: true

  # Repository delegations
  delegate :acronym, to: :repository, prefix: :repository, allow_nil: true
  delegate :url, to: :repository, prefix: :repository, allow_nil: true
  delegate :institutional_LSID, to: :repository, prefix: :repository, allow_nil: true

  # Preparation delegations
  delegate :name, to: :preparation_type, prefix: :preparation_type, allow_nil: true

  has_one :accession_provider_role, class_name: 'AccessionProvider', as: :role_object, dependent: :destroy
  has_one :accession_provider, through: :accession_provider_role, source: :person
  has_one :deaccession_recipient_role, class_name: 'DeaccessionRecipient', as: :role_object, dependent: :destroy
  has_one :deaccession_recipient, through: :deaccession_recipient_role, source: :person

  # TODO: Deprecate these models.  Semantics also confuse with origin relationship.
  has_many :derived_collection_objects, inverse_of: :collection_object, dependent: :restrict_with_error
  has_many :collection_object_observations, through: :derived_collection_objects, inverse_of: :collection_objects

  has_many :sqed_depictions, through: :depictions, dependent: :restrict_with_error

  belongs_to :collecting_event, inverse_of: :collection_objects
  belongs_to :preparation_type, inverse_of: :collection_objects
  belongs_to :ranged_lot_category, inverse_of: :ranged_lots
  belongs_to :repository, inverse_of: :collection_objects
  belongs_to :current_repository, class_name: 'Repository', inverse_of: :collection_objects

  has_many :georeferences, through: :collecting_event
  has_many :geographic_items, through: :georeferences

  has_many :collectors, through: :collecting_event

  accepts_nested_attributes_for :collecting_event, allow_destroy: true, reject_if: :reject_collecting_event

  before_validation :assign_type_if_total_or_ranged_lot_category_id_provided

  validates_presence_of :type
  validate :check_that_either_total_or_ranged_lot_category_id_is_present
  validate :check_that_both_of_category_and_total_are_not_present
  validate :collecting_event_belongs_to_project

  soft_validate(
    :sv_missing_accession_fields,
    set: :missing_accession_fields,
    name: 'Missing accession fields',
    description: 'Name or Provider are not selected')

  soft_validate(
    :sv_missing_deaccession_fields,
    set: :missing_deaccession_fields,
    name: 'Missing deaccesson fields',
    description: 'Date, recipient, or reason are not specified')

  scope :with_sequence_name, ->(name) { joins(sequence_join_hack_sql).where(sequences: {name:}) }
  scope :via_descriptor, ->(descriptor) { joins(sequence_join_hack_sql).where(sequences: {id: descriptor.sequences}) }

  has_many :extracts, through: :origin_relationships, source: :new_object, source_type: 'Extract'
  has_many :sequences, through: :extracts

  # This is a hack, maybe related to a Rails 5.1 bug.
  # It returns the SQL that works in 5.0/4.2 that
  # links CollectionObject to Sequences:
  # joins(derived_extracts: [:derived_sequences])
  def self.sequence_join_hack_sql
    %Q{INNER JOIN  "origin_relationships"
               ON  "origin_relationships"."old_object_id" = "collection_objects"."id"
                  AND  "origin_relationships"."new_object_type" = 'Extract'
                  AND  "origin_relationships"."old_object_type" = 'CollectionObject'
       INNER JOIN  "extracts"
               ON  "extracts"."id" =  "origin_relationships"."new_object_id"
       INNER JOIN  "origin_relationships" "origin_relationships_extracts_join"
               ON  "origin_relationships_extracts_join"."old_object_id" = "extracts"."id"
                  AND  "origin_relationships_extracts_join"."new_object_type" = 'Sequence'
                  AND  "origin_relationships_extracts_join"."old_object_type" = 'Extract'
       INNER JOIN  "sequences"
               ON  "sequences"."id" = "origin_relationships_extracts_join"."new_object_id"}
  end

  def self.batch_update(params)
    request = QueryBatchRequest.new(
      async_cutoff: params[:async_cutoff] || 50,
      klass: 'CollectionObject',
      object_filter_params: params[:collection_object_query],
      object_params: params[:collection_object],
      preview: params[:preview],
    )

    request.cap = 1000

    query_batch_update(request)
  end

  def self.batch_update_dwc_occurrence(params)
    q = Queries::CollectionObject::Filter.new(params).all

    r = BatchResponse.new
    r.method = 'batch_update_dwc_occurrence'
    r.klass = 'CollectionObject'

    c = q.all.count

    if c == 0 || c > 10000
      r.cap_reason = 'Too many (or no) collection objects (max 10k)'
      return r
    end

    if c < 51
      q.each do |co|
        co.set_dwc_occurrence
        r.updated.push co.id
      end
    else
      r.async = true
      q.each do |co|
        co.dwc_occurrence_update_query
      end
    end

    return r
  end

  def dwc_occurrence_update_query
    self.send(:set_dwc_occurrence)
  end

  handle_asynchronously :dwc_occurrence_update_query, run_at: Proc.new { 1.second.from_now }, queue: :query_batch_update

  # TODO: move to a helper
  def self.breakdown_status(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array

    breakdown = {
      total_objects:     collection_objects.length,
      collecting_events: {},
      determinations:    {},
      bio_overview:      []
    }

    breakdown.merge!(breakdown_buffered(collection_objects))

    collection_objects.each do |co|
      breakdown[:collecting_events].merge!(co => co.collecting_event) if co.collecting_event
      breakdown[:determinations].merge!(co => co.taxon_determinations) if co.taxon_determinations.load.any?
      breakdown[:bio_overview].push([co.total, co.biocuration_classes.collect { |a| a.name }])
    end

    breakdown
  end

  # @return [Hash]
  #   a unque list of buffered_ values observed in the collection objects passed
  def self.breakdown_buffered(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array
    breakdown = {}
    categories = BUFFERED_ATTRIBUTES

    categories.each do |c|
      breakdown[c] = []
    end

    categories.each do |c|
      collection_objects.each do |co|
        breakdown[c].push co.send(c)
      end
    end

    categories.each do |c|
      breakdown[c].uniq!
    end

    breakdown
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.earliest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:end_date_year)

    return EARLIEST_DATE if a.nil? && b.nil?  # 1700-01-01

    d = nil

    if a && b
      if a < b
        d = a
      end
    else
      d = a || b
    end
    d.to_s + '-01-01'
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.latest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:end_date_year)

    c = Time.now.strftime('%Y-%m-%d')

    return c if a.nil? && b.nil?

    d = nil

    if a && b
      if a > b
        d = a
      end
    else
      d = a || b
    end

    d.to_s + '/12/31'
  end

  # TODO: Clarify this.
  # CAREFULL - this isn't _in_, this is *with*, if it was in it would be spatial query, not a join(:geographic_items)
  #
  # Find all collection objects which have collecting events which have georeferences which have geographic_items which
  # are located within the geographic item supplied
  # @param [GeographicItem] geographic_item_id
  # @return [Scope] of CollectionObject
  def self.in_geographic_item(geographic_item, limit, steps = false)
    geographic_item_id = geographic_item.id
    if steps
      gi = GeographicItem.find(geographic_item_id)
      # find the geographic_items inside gi
      step_1 = GeographicItem.is_contained_by('any', gi) # .pluck(:id)
      # find the georeferences from the geographic_items
      step_2 = step_1.map(&:georeferences).uniq.flatten
      # find the collecting events connected to the georeferences
      step_3 = step_2.map(&:collecting_event).uniq.flatten
      # find the collection objects associated with the collecting events
      step_4 = step_3.map(&:collection_objects).flatten.map(&:id).uniq
      retval = CollectionObject.where(id: step_4.sort)
    else
      retval = CollectionObject.joins(:geographic_items)
        .where(GeographicItem.contained_by_where_sql(geographic_item.id))
        .limit(limit)
        .includes(:data_attributes, :collecting_event)
    end
    retval
  end

  # TODO: deprecate
  def self.selected_column_names
    @selected_column_names = {
      ce: {in: {}, im: {}},
      co: {in: {}, im: {}},
      bc: {in: {}, im: {}}
    } if @selected_column_names.nil?
    @selected_column_names
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collecting events
  # decode which headers to be displayed for collecting events
  def self.ce_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:ce][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:ce][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.ce_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.collecting_event.internal_attributes
      all_import_das   = collection_object.collecting_event.import_attributes
      group            = collection[:ce]
      unless group.nil?
        group.each_key { |type_key|
          group[type_key.to_sym].each_key { |header|
            this_val = nil
            case type_key.to_sym
            when :in
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            when :im
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            else
            end
          }
        }
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collection objects
  # decode which headers to be displayed for collection objects
  def self.co_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:co][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:co][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.co_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.internal_attributes
      all_import_das   = collection_object.import_attributes
      group            = collection[:co]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = nil
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
        unless group.empty?
          unless group[:im].empty?
            group[:im].each_key { |header|
              this_val = nil
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for biocuration classifications
  # decode which headers to be displayed for biocuration classifications
  def self.bc_headers(project_id)
    CollectionObject.selected_column_names
    # add selectable column names (unselected) to the column name list list
    BiocurationClass.where(project_id:).map(&:name).each { |column_name|
      @selected_column_names[:bc][:in][column_name] = {checked: '0'}
    }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.bc_attributes(collection_object, col_defs)
    retval = []
    collection = col_defs
    unless collection.nil?
      group = collection[:bc]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = collection_object.biocuration_classes.map(&:name).include?(header) ? '1' : '0'
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Array] collecting_event_ids (e.g., from CollectingEvent.in_date_range)
  # @param [Array] area_object_ids (e.g., from GeographicItem.gather_selected_data())
  # @return [Scope] of intersection of collecting events (usually by date range)
  #   and collection objects (usually by inclusion in geographic areas/items)
  def self.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id)
    collecting_events_clause = {collecting_event_id: collecting_event_ids, project: project_id}
    area_objects_clause      = {id: area_object_ids, project: project_id}

    if (collecting_event_ids.empty?)
      collecting_events_clause = {project: project_id}
    end

    if (area_object_ids.empty?)
      area_objects_clause = {}
      if (area_set)
        area_objects_clause = 'false'
      end
    end

    retval = CollectionObject.joins(:collecting_event)
      .where(collecting_events_clause)
      .where(area_objects_clause)
    retval
  end

  # TODO: move to filter
  # @param [Hash] search_start_date string in form 'yyyy-mm-dd'
  # @param [Hash] search_end_date string in form 'yyyy-mm-dd'
  # @param [Hash] partial_overlap 'on' or 'off'
  # @return [Scope] of selected collection objects through collecting events with georeferences, remember to scope to project!
  def self.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on')
    allow_partial = (partial_overlap.downcase == 'off' ? false : true) # TODO: Just get the correct values from the form!
    q = Queries::CollectingEvent::Filter.new(start_date: search_start_date, end_date: search_end_date, partial_overlap_dates: allow_partial)
    joins(:collecting_event).where(q.between_date_range_facet.to_sql)
  end

  # @param used_on [String] required, one of `TaxonDetermination`, `BiologicalAssociation`
  # @return [Scope]
  #    the max 10 most recently used collection_objects, as `used_on`
  def self.used_recently(user_id, project_id, used_on = '')
    return [] if used_on != 'TaxonDetermination' && used_on != 'BiologicalAssociation'
    t = case used_on
        when 'TaxonDetermination'
          TaxonDetermination.arel_table
        when 'BiologicalAssociation'
          BiologicalAssociation.arel_table
        end

    p = CollectionObject.arel_table

    # i is a select manager
    i = case used_on
        when 'BiologicalAssociation'
          t.project(t['biological_association_subject_id'], t['updated_at']).from(t)
            .where(
              t['updated_at'].gt(1.week.ago).and(
                t['biological_association_subject_type'].eq('CollectionObject')
              )
            )
              .where(t['updated_by_id'].eq(user_id))
              .where(t['project_id'].eq(project_id))
              .order(t['updated_at'].desc)
        else
          # TODO: update to reference new TaxonDetermination
          t.project(t['taxon_determination_object_id'], t['taxon_determination_object_type'], t['updated_at']).from(t)
            .where(t['updated_at'].gt( 1.week.ago ))
            .where(t['updated_by_id'].eq(user_id))
            .where(t['project_id'].eq(project_id))
            .order(t['updated_at'].desc)
        end

    # z is a table alias
    z = i.as('recent_t')

    j = case used_on
        when 'BiologicalAssociation'
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(
            z['biological_association_subject_id'].eq(p['id'])
          ))
        else
          # TODO: needs to be fixed to scope the taxon_determination_object_type
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(z['taxon_determination_object_id'].eq(p['id'])))
        end

    CollectionObject.joins(j).pluck(:id).uniq
  end

  # @params target [String] one of `TaxonDetermination`, `BiologicalAssociation` , nil
  # @return [Hash] otus optimized for user selection
  def self.select_optimized(user_id, project_id, target = nil)
    r = used_recently(user_id, project_id, target)
    h = {
      quick: [],
      pinboard: CollectionObject.pinned_by(user_id).where(project_id:).to_a,
      recent: []
    }

    if target && !r.empty?
      n = target.tableize.to_sym
      h[:recent] = CollectionObject.where('"collection_objects"."id" IN (?)', r.first(10) ).to_a
      h[:quick] = (CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a  +
                   CollectionObject.where('"collection_objects"."id" IN (?)', r.first(4) ).to_a).uniq
    else
      h[:recent] = CollectionObject.where(project_id:, updated_by_id: user_id).order('updated_at DESC').limit(10).to_a
      h[:quick] = CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a
    end

    h
  end

  # TODO: Unify with Extract in concern
  # @return [Identifier::Local::CatalogNumber, nil]
  #   the first (position) catalog number for this collection object, either on specimen, or container
  def preferred_catalog_number
    if i = Identifier::Local::CatalogNumber.where(identifier_object: self).order(:position).first
      i
    else
      if container
        container.identifiers.where(identifiers: {type: 'Identifier::Local::CatalogNumber'}).order(:position).first
      else
        nil
      end
    end
  end

  # @return [Identifier::Local::RecordNumber, nil]
  #   the first (position) record_Number, on a specimen
  #   !1 Doesn't presently support containers
  def preferred_record_number
    Identifier::Local::RecordNumber.where(identifier_object: self).order(:position).first
  end

  def geographic_name_classification
    # don't load the whole object, just the fields we need
    if a = DwcOccurrence.where(dwc_occurrence_object: self).select(:country, :stateProvince, :county).first

      c = a.country
      s = a.stateProvince
      y = a.county

      v = ::Utilities::Geo::DICTIONARY[c]
      c = v if v
      # s = v if v = ::Utilities::Geo::DICTIONARY[s] # None in there yet
      # y = v if v = ::Utilities::Geo::DICTIONARY[y] # None in there yet

      return {
        country: c,
        state: s,
        county: y
      }
    end
  end

  # return [Boolean]
  #    True if instance is a subclass of BiologicalCollectionObject
  def is_biological?
    self.class <= BiologicalCollectionObject ? true : false
  end

  def annotations
    h = annotations_hash
    (h['biocuration classifications'] = biocuration_classes) if is_biological? && biocuration_classifications.load.any?
    h
  end

  def sv_missing_accession_fields
    soft_validations.add(:accessioned_at, 'Date is not selected') if self.accessioned_at.nil? && !self.accession_provider.nil?
    soft_validations.add(:base, 'Provider is not selected') if !self.accessioned_at.nil? && self.accession_provider.nil?
  end

  def sv_missing_deaccession_fields
    soft_validations.add(:deaccessioned_at, 'Date is not selected') if self.deaccessioned_at.nil? && self.deaccession_reason.present?
    soft_validations.add(:base, 'Recipient is not selected') if self.deaccession_recipient.nil? && self.deaccession_reason && self.deaccessioned_at
    soft_validations.add(:deaccession_reason, 'Reason is is not defined') if self.deaccession_reason.blank? && self.deaccession_recipient && self.deaccessioned_at
  end

  def sv_missing_determination
    # see biological_collection_object
  end

  def sv_missing_collecting_event
    # see biological_collection_object
  end

  def sv_missing_preparation_type
    # see biological_collection_object
  end

  def sv_missing_repository
    # WHY? -  see biological_collection_object
  end

  def sv_missing_biocuration_classification
    # see biological_collection_object
  end

  # See Depiction#destroy_image_stub_collection_object
  # Used to determin if the CO can be
  # destroy after moving an image off
  # this object.
  def is_image_stub?
    r = [
      collecting_event_id.blank?,
      !depictions.reload.any?,
      identifiers.count <= 1,
      !taxon_determinations.any?,
      !type_materials.any?,
      !citations.any?,
      !data_attributes.any?,
      !notes.any?,
      !observations.any?
    ]

   !r.include?(false)

  end

  protected

  def collecting_event_belongs_to_project
    if collecting_event&.persisted? && (Current.project_id || project_id)
      errors.add(:base, 'collecting event is not from this project') if collecting_event.project_id != (Current.project_id || project_id)
    end
  end

  def check_that_both_of_category_and_total_are_not_present
    errors.add(:ranged_lot_category_id, 'Both ranged_lot_category and total can not be set') if ranged_lot_category_id.present? && total.present?
  end

  def check_that_either_total_or_ranged_lot_category_id_is_present
    errors.add(:base, 'Either total or a ranged lot category must be provided') if ranged_lot_category_id.blank? && total.blank?
  end

  def assign_type_if_total_or_ranged_lot_category_id_provided
    if self.total == 1
      self.type = 'Specimen'
    elsif self.total.to_i > 1
      self.type = 'Lot'
    elsif total.nil? && ranged_lot_category_id.present?
      self.type = 'RangedLot'
    end
    true
  end

  def reject_collecting_event(attributed)
    reject = true
    CollectingEvent.core_attributes.each do |a|
      if attributed[a].present?
        reject = false
        break
      end
    end
    # !! does not account for georeferences_attributes!
    reject
  end

end

#deaccession_reason ⇒ String

A free text explanation of why the object was removed from tracking.

Returns:

• (String)

# File 'app/models/collection_object.rb', line 64

class CollectionObject < ApplicationRecord
  include GlobalID::Identification
  include Housekeeping

  include Shared::Citations
  include Shared::Containable
  include Shared::DataAttributes
  include Shared::Loanable
  include Shared::Identifiers
  include Shared::Notes
  include Shared::Tags
  include Shared::Depictions
  include Shared::OriginRelationship
  include Shared::Confidences
  include Shared::ProtocolRelationships
  include Shared::HasPapertrail
  include Shared::Observations
  include Shared::IsData
  include Shared::QueryBatchUpdate
  include SoftValidation

  include Shared::BiologicalExtensions

  include Shared::Taxonomy # at present must be before IsDwcOccurence
  include Shared::IsDwcOccurrence
  include CollectionObject::DwcExtensions

  ignore_whitespace_on(:buffered_collecting_event, :buffered_determinations, :buffered_other_labels)

  # TODO: move to export
  CO_OTU_HEADERS = %w{OTU OTU\ name Family Genus Species Country State County Locality Latitude Longitude}.freeze

  BUFFERED_ATTRIBUTES = %i{buffered_collecting_event buffered_determinations buffered_other_labels}.freeze

  GRAPH_ENTRY_POINTS = [:biological_associations, :data_attributes, :taxon_determinations, :biocuration_classifications, :collecting_event, :origin_relationships, :extracts, :observation_matrices]

  # Identifier delegations
  # .catalog_number_cached
  delegate :cached, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true
  # .catalog_number_namespace
  delegate :namespace, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true

  # .record_number_cached
  delegate :cached, to: :preferred_record_number, prefix: :record_number, allow_nil: true
  # .record_number_namespace
  delegate :namespace, to: :preferred_record_number, prefix: :record_number, allow_nil: true

  # CollectingEvent delegations
  delegate :map_center, to: :collecting_event, prefix: :collecting_event, allow_nil: true
  delegate :collectors, to: :collecting_event, prefix: :collecting_event, allow_nil: true

  # Repository delegations
  delegate :acronym, to: :repository, prefix: :repository, allow_nil: true
  delegate :url, to: :repository, prefix: :repository, allow_nil: true
  delegate :institutional_LSID, to: :repository, prefix: :repository, allow_nil: true

  # Preparation delegations
  delegate :name, to: :preparation_type, prefix: :preparation_type, allow_nil: true

  has_one :accession_provider_role, class_name: 'AccessionProvider', as: :role_object, dependent: :destroy
  has_one :accession_provider, through: :accession_provider_role, source: :person
  has_one :deaccession_recipient_role, class_name: 'DeaccessionRecipient', as: :role_object, dependent: :destroy
  has_one :deaccession_recipient, through: :deaccession_recipient_role, source: :person

  # TODO: Deprecate these models.  Semantics also confuse with origin relationship.
  has_many :derived_collection_objects, inverse_of: :collection_object, dependent: :restrict_with_error
  has_many :collection_object_observations, through: :derived_collection_objects, inverse_of: :collection_objects

  has_many :sqed_depictions, through: :depictions, dependent: :restrict_with_error

  belongs_to :collecting_event, inverse_of: :collection_objects
  belongs_to :preparation_type, inverse_of: :collection_objects
  belongs_to :ranged_lot_category, inverse_of: :ranged_lots
  belongs_to :repository, inverse_of: :collection_objects
  belongs_to :current_repository, class_name: 'Repository', inverse_of: :collection_objects

  has_many :georeferences, through: :collecting_event
  has_many :geographic_items, through: :georeferences

  has_many :collectors, through: :collecting_event

  accepts_nested_attributes_for :collecting_event, allow_destroy: true, reject_if: :reject_collecting_event

  before_validation :assign_type_if_total_or_ranged_lot_category_id_provided

  validates_presence_of :type
  validate :check_that_either_total_or_ranged_lot_category_id_is_present
  validate :check_that_both_of_category_and_total_are_not_present
  validate :collecting_event_belongs_to_project

  soft_validate(
    :sv_missing_accession_fields,
    set: :missing_accession_fields,
    name: 'Missing accession fields',
    description: 'Name or Provider are not selected')

  soft_validate(
    :sv_missing_deaccession_fields,
    set: :missing_deaccession_fields,
    name: 'Missing deaccesson fields',
    description: 'Date, recipient, or reason are not specified')

  scope :with_sequence_name, ->(name) { joins(sequence_join_hack_sql).where(sequences: {name:}) }
  scope :via_descriptor, ->(descriptor) { joins(sequence_join_hack_sql).where(sequences: {id: descriptor.sequences}) }

  has_many :extracts, through: :origin_relationships, source: :new_object, source_type: 'Extract'
  has_many :sequences, through: :extracts

  # This is a hack, maybe related to a Rails 5.1 bug.
  # It returns the SQL that works in 5.0/4.2 that
  # links CollectionObject to Sequences:
  # joins(derived_extracts: [:derived_sequences])
  def self.sequence_join_hack_sql
    %Q{INNER JOIN  "origin_relationships"
               ON  "origin_relationships"."old_object_id" = "collection_objects"."id"
                  AND  "origin_relationships"."new_object_type" = 'Extract'
                  AND  "origin_relationships"."old_object_type" = 'CollectionObject'
       INNER JOIN  "extracts"
               ON  "extracts"."id" =  "origin_relationships"."new_object_id"
       INNER JOIN  "origin_relationships" "origin_relationships_extracts_join"
               ON  "origin_relationships_extracts_join"."old_object_id" = "extracts"."id"
                  AND  "origin_relationships_extracts_join"."new_object_type" = 'Sequence'
                  AND  "origin_relationships_extracts_join"."old_object_type" = 'Extract'
       INNER JOIN  "sequences"
               ON  "sequences"."id" = "origin_relationships_extracts_join"."new_object_id"}
  end

  def self.batch_update(params)
    request = QueryBatchRequest.new(
      async_cutoff: params[:async_cutoff] || 50,
      klass: 'CollectionObject',
      object_filter_params: params[:collection_object_query],
      object_params: params[:collection_object],
      preview: params[:preview],
    )

    request.cap = 1000

    query_batch_update(request)
  end

  def self.batch_update_dwc_occurrence(params)
    q = Queries::CollectionObject::Filter.new(params).all

    r = BatchResponse.new
    r.method = 'batch_update_dwc_occurrence'
    r.klass = 'CollectionObject'

    c = q.all.count

    if c == 0 || c > 10000
      r.cap_reason = 'Too many (or no) collection objects (max 10k)'
      return r
    end

    if c < 51
      q.each do |co|
        co.set_dwc_occurrence
        r.updated.push co.id
      end
    else
      r.async = true
      q.each do |co|
        co.dwc_occurrence_update_query
      end
    end

    return r
  end

  def dwc_occurrence_update_query
    self.send(:set_dwc_occurrence)
  end

  handle_asynchronously :dwc_occurrence_update_query, run_at: Proc.new { 1.second.from_now }, queue: :query_batch_update

  # TODO: move to a helper
  def self.breakdown_status(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array

    breakdown = {
      total_objects:     collection_objects.length,
      collecting_events: {},
      determinations:    {},
      bio_overview:      []
    }

    breakdown.merge!(breakdown_buffered(collection_objects))

    collection_objects.each do |co|
      breakdown[:collecting_events].merge!(co => co.collecting_event) if co.collecting_event
      breakdown[:determinations].merge!(co => co.taxon_determinations) if co.taxon_determinations.load.any?
      breakdown[:bio_overview].push([co.total, co.biocuration_classes.collect { |a| a.name }])
    end

    breakdown
  end

  # @return [Hash]
  #   a unque list of buffered_ values observed in the collection objects passed
  def self.breakdown_buffered(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array
    breakdown = {}
    categories = BUFFERED_ATTRIBUTES

    categories.each do |c|
      breakdown[c] = []
    end

    categories.each do |c|
      collection_objects.each do |co|
        breakdown[c].push co.send(c)
      end
    end

    categories.each do |c|
      breakdown[c].uniq!
    end

    breakdown
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.earliest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:end_date_year)

    return EARLIEST_DATE if a.nil? && b.nil?  # 1700-01-01

    d = nil

    if a && b
      if a < b
        d = a
      end
    else
      d = a || b
    end
    d.to_s + '-01-01'
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.latest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:end_date_year)

    c = Time.now.strftime('%Y-%m-%d')

    return c if a.nil? && b.nil?

    d = nil

    if a && b
      if a > b
        d = a
      end
    else
      d = a || b
    end

    d.to_s + '/12/31'
  end

  # TODO: Clarify this.
  # CAREFULL - this isn't _in_, this is *with*, if it was in it would be spatial query, not a join(:geographic_items)
  #
  # Find all collection objects which have collecting events which have georeferences which have geographic_items which
  # are located within the geographic item supplied
  # @param [GeographicItem] geographic_item_id
  # @return [Scope] of CollectionObject
  def self.in_geographic_item(geographic_item, limit, steps = false)
    geographic_item_id = geographic_item.id
    if steps
      gi = GeographicItem.find(geographic_item_id)
      # find the geographic_items inside gi
      step_1 = GeographicItem.is_contained_by('any', gi) # .pluck(:id)
      # find the georeferences from the geographic_items
      step_2 = step_1.map(&:georeferences).uniq.flatten
      # find the collecting events connected to the georeferences
      step_3 = step_2.map(&:collecting_event).uniq.flatten
      # find the collection objects associated with the collecting events
      step_4 = step_3.map(&:collection_objects).flatten.map(&:id).uniq
      retval = CollectionObject.where(id: step_4.sort)
    else
      retval = CollectionObject.joins(:geographic_items)
        .where(GeographicItem.contained_by_where_sql(geographic_item.id))
        .limit(limit)
        .includes(:data_attributes, :collecting_event)
    end
    retval
  end

  # TODO: deprecate
  def self.selected_column_names
    @selected_column_names = {
      ce: {in: {}, im: {}},
      co: {in: {}, im: {}},
      bc: {in: {}, im: {}}
    } if @selected_column_names.nil?
    @selected_column_names
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collecting events
  # decode which headers to be displayed for collecting events
  def self.ce_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:ce][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:ce][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.ce_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.collecting_event.internal_attributes
      all_import_das   = collection_object.collecting_event.import_attributes
      group            = collection[:ce]
      unless group.nil?
        group.each_key { |type_key|
          group[type_key.to_sym].each_key { |header|
            this_val = nil
            case type_key.to_sym
            when :in
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            when :im
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            else
            end
          }
        }
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collection objects
  # decode which headers to be displayed for collection objects
  def self.co_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:co][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:co][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.co_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.internal_attributes
      all_import_das   = collection_object.import_attributes
      group            = collection[:co]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = nil
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
        unless group.empty?
          unless group[:im].empty?
            group[:im].each_key { |header|
              this_val = nil
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for biocuration classifications
  # decode which headers to be displayed for biocuration classifications
  def self.bc_headers(project_id)
    CollectionObject.selected_column_names
    # add selectable column names (unselected) to the column name list list
    BiocurationClass.where(project_id:).map(&:name).each { |column_name|
      @selected_column_names[:bc][:in][column_name] = {checked: '0'}
    }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.bc_attributes(collection_object, col_defs)
    retval = []
    collection = col_defs
    unless collection.nil?
      group = collection[:bc]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = collection_object.biocuration_classes.map(&:name).include?(header) ? '1' : '0'
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Array] collecting_event_ids (e.g., from CollectingEvent.in_date_range)
  # @param [Array] area_object_ids (e.g., from GeographicItem.gather_selected_data())
  # @return [Scope] of intersection of collecting events (usually by date range)
  #   and collection objects (usually by inclusion in geographic areas/items)
  def self.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id)
    collecting_events_clause = {collecting_event_id: collecting_event_ids, project: project_id}
    area_objects_clause      = {id: area_object_ids, project: project_id}

    if (collecting_event_ids.empty?)
      collecting_events_clause = {project: project_id}
    end

    if (area_object_ids.empty?)
      area_objects_clause = {}
      if (area_set)
        area_objects_clause = 'false'
      end
    end

    retval = CollectionObject.joins(:collecting_event)
      .where(collecting_events_clause)
      .where(area_objects_clause)
    retval
  end

  # TODO: move to filter
  # @param [Hash] search_start_date string in form 'yyyy-mm-dd'
  # @param [Hash] search_end_date string in form 'yyyy-mm-dd'
  # @param [Hash] partial_overlap 'on' or 'off'
  # @return [Scope] of selected collection objects through collecting events with georeferences, remember to scope to project!
  def self.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on')
    allow_partial = (partial_overlap.downcase == 'off' ? false : true) # TODO: Just get the correct values from the form!
    q = Queries::CollectingEvent::Filter.new(start_date: search_start_date, end_date: search_end_date, partial_overlap_dates: allow_partial)
    joins(:collecting_event).where(q.between_date_range_facet.to_sql)
  end

  # @param used_on [String] required, one of `TaxonDetermination`, `BiologicalAssociation`
  # @return [Scope]
  #    the max 10 most recently used collection_objects, as `used_on`
  def self.used_recently(user_id, project_id, used_on = '')
    return [] if used_on != 'TaxonDetermination' && used_on != 'BiologicalAssociation'
    t = case used_on
        when 'TaxonDetermination'
          TaxonDetermination.arel_table
        when 'BiologicalAssociation'
          BiologicalAssociation.arel_table
        end

    p = CollectionObject.arel_table

    # i is a select manager
    i = case used_on
        when 'BiologicalAssociation'
          t.project(t['biological_association_subject_id'], t['updated_at']).from(t)
            .where(
              t['updated_at'].gt(1.week.ago).and(
                t['biological_association_subject_type'].eq('CollectionObject')
              )
            )
              .where(t['updated_by_id'].eq(user_id))
              .where(t['project_id'].eq(project_id))
              .order(t['updated_at'].desc)
        else
          # TODO: update to reference new TaxonDetermination
          t.project(t['taxon_determination_object_id'], t['taxon_determination_object_type'], t['updated_at']).from(t)
            .where(t['updated_at'].gt( 1.week.ago ))
            .where(t['updated_by_id'].eq(user_id))
            .where(t['project_id'].eq(project_id))
            .order(t['updated_at'].desc)
        end

    # z is a table alias
    z = i.as('recent_t')

    j = case used_on
        when 'BiologicalAssociation'
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(
            z['biological_association_subject_id'].eq(p['id'])
          ))
        else
          # TODO: needs to be fixed to scope the taxon_determination_object_type
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(z['taxon_determination_object_id'].eq(p['id'])))
        end

    CollectionObject.joins(j).pluck(:id).uniq
  end

  # @params target [String] one of `TaxonDetermination`, `BiologicalAssociation` , nil
  # @return [Hash] otus optimized for user selection
  def self.select_optimized(user_id, project_id, target = nil)
    r = used_recently(user_id, project_id, target)
    h = {
      quick: [],
      pinboard: CollectionObject.pinned_by(user_id).where(project_id:).to_a,
      recent: []
    }

    if target && !r.empty?
      n = target.tableize.to_sym
      h[:recent] = CollectionObject.where('"collection_objects"."id" IN (?)', r.first(10) ).to_a
      h[:quick] = (CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a  +
                   CollectionObject.where('"collection_objects"."id" IN (?)', r.first(4) ).to_a).uniq
    else
      h[:recent] = CollectionObject.where(project_id:, updated_by_id: user_id).order('updated_at DESC').limit(10).to_a
      h[:quick] = CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a
    end

    h
  end

  # TODO: Unify with Extract in concern
  # @return [Identifier::Local::CatalogNumber, nil]
  #   the first (position) catalog number for this collection object, either on specimen, or container
  def preferred_catalog_number
    if i = Identifier::Local::CatalogNumber.where(identifier_object: self).order(:position).first
      i
    else
      if container
        container.identifiers.where(identifiers: {type: 'Identifier::Local::CatalogNumber'}).order(:position).first
      else
        nil
      end
    end
  end

  # @return [Identifier::Local::RecordNumber, nil]
  #   the first (position) record_Number, on a specimen
  #   !1 Doesn't presently support containers
  def preferred_record_number
    Identifier::Local::RecordNumber.where(identifier_object: self).order(:position).first
  end

  def geographic_name_classification
    # don't load the whole object, just the fields we need
    if a = DwcOccurrence.where(dwc_occurrence_object: self).select(:country, :stateProvince, :county).first

      c = a.country
      s = a.stateProvince
      y = a.county

      v = ::Utilities::Geo::DICTIONARY[c]
      c = v if v
      # s = v if v = ::Utilities::Geo::DICTIONARY[s] # None in there yet
      # y = v if v = ::Utilities::Geo::DICTIONARY[y] # None in there yet

      return {
        country: c,
        state: s,
        county: y
      }
    end
  end

  # return [Boolean]
  #    True if instance is a subclass of BiologicalCollectionObject
  def is_biological?
    self.class <= BiologicalCollectionObject ? true : false
  end

  def annotations
    h = annotations_hash
    (h['biocuration classifications'] = biocuration_classes) if is_biological? && biocuration_classifications.load.any?
    h
  end

  def sv_missing_accession_fields
    soft_validations.add(:accessioned_at, 'Date is not selected') if self.accessioned_at.nil? && !self.accession_provider.nil?
    soft_validations.add(:base, 'Provider is not selected') if !self.accessioned_at.nil? && self.accession_provider.nil?
  end

  def sv_missing_deaccession_fields
    soft_validations.add(:deaccessioned_at, 'Date is not selected') if self.deaccessioned_at.nil? && self.deaccession_reason.present?
    soft_validations.add(:base, 'Recipient is not selected') if self.deaccession_recipient.nil? && self.deaccession_reason && self.deaccessioned_at
    soft_validations.add(:deaccession_reason, 'Reason is is not defined') if self.deaccession_reason.blank? && self.deaccession_recipient && self.deaccessioned_at
  end

  def sv_missing_determination
    # see biological_collection_object
  end

  def sv_missing_collecting_event
    # see biological_collection_object
  end

  def sv_missing_preparation_type
    # see biological_collection_object
  end

  def sv_missing_repository
    # WHY? -  see biological_collection_object
  end

  def sv_missing_biocuration_classification
    # see biological_collection_object
  end

  # See Depiction#destroy_image_stub_collection_object
  # Used to determin if the CO can be
  # destroy after moving an image off
  # this object.
  def is_image_stub?
    r = [
      collecting_event_id.blank?,
      !depictions.reload.any?,
      identifiers.count <= 1,
      !taxon_determinations.any?,
      !type_materials.any?,
      !citations.any?,
      !data_attributes.any?,
      !notes.any?,
      !observations.any?
    ]

   !r.include?(false)

  end

  protected

  def collecting_event_belongs_to_project
    if collecting_event&.persisted? && (Current.project_id || project_id)
      errors.add(:base, 'collecting event is not from this project') if collecting_event.project_id != (Current.project_id || project_id)
    end
  end

  def check_that_both_of_category_and_total_are_not_present
    errors.add(:ranged_lot_category_id, 'Both ranged_lot_category and total can not be set') if ranged_lot_category_id.present? && total.present?
  end

  def check_that_either_total_or_ranged_lot_category_id_is_present
    errors.add(:base, 'Either total or a ranged lot category must be provided') if ranged_lot_category_id.blank? && total.blank?
  end

  def assign_type_if_total_or_ranged_lot_category_id_provided
    if self.total == 1
      self.type = 'Specimen'
    elsif self.total.to_i > 1
      self.type = 'Lot'
    elsif total.nil? && ranged_lot_category_id.present?
      self.type = 'RangedLot'
    end
    true
  end

  def reject_collecting_event(attributed)
    reject = true
    CollectingEvent.core_attributes.each do |a|
      if attributed[a].present?
        reject = false
        break
      end
    end
    # !! does not account for georeferences_attributes!
    reject
  end

end

#deaccessioned_at ⇒ Date

The date when the object was removed from tracking. If provide then Repository must be null?! TODO: resolve

Returns:

• (Date)

# File 'app/models/collection_object.rb', line 64

class CollectionObject < ApplicationRecord
  include GlobalID::Identification
  include Housekeeping

  include Shared::Citations
  include Shared::Containable
  include Shared::DataAttributes
  include Shared::Loanable
  include Shared::Identifiers
  include Shared::Notes
  include Shared::Tags
  include Shared::Depictions
  include Shared::OriginRelationship
  include Shared::Confidences
  include Shared::ProtocolRelationships
  include Shared::HasPapertrail
  include Shared::Observations
  include Shared::IsData
  include Shared::QueryBatchUpdate
  include SoftValidation

  include Shared::BiologicalExtensions

  include Shared::Taxonomy # at present must be before IsDwcOccurence
  include Shared::IsDwcOccurrence
  include CollectionObject::DwcExtensions

  ignore_whitespace_on(:buffered_collecting_event, :buffered_determinations, :buffered_other_labels)

  # TODO: move to export
  CO_OTU_HEADERS = %w{OTU OTU\ name Family Genus Species Country State County Locality Latitude Longitude}.freeze

  BUFFERED_ATTRIBUTES = %i{buffered_collecting_event buffered_determinations buffered_other_labels}.freeze

  GRAPH_ENTRY_POINTS = [:biological_associations, :data_attributes, :taxon_determinations, :biocuration_classifications, :collecting_event, :origin_relationships, :extracts, :observation_matrices]

  # Identifier delegations
  # .catalog_number_cached
  delegate :cached, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true
  # .catalog_number_namespace
  delegate :namespace, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true

  # .record_number_cached
  delegate :cached, to: :preferred_record_number, prefix: :record_number, allow_nil: true
  # .record_number_namespace
  delegate :namespace, to: :preferred_record_number, prefix: :record_number, allow_nil: true

  # CollectingEvent delegations
  delegate :map_center, to: :collecting_event, prefix: :collecting_event, allow_nil: true
  delegate :collectors, to: :collecting_event, prefix: :collecting_event, allow_nil: true

  # Repository delegations
  delegate :acronym, to: :repository, prefix: :repository, allow_nil: true
  delegate :url, to: :repository, prefix: :repository, allow_nil: true
  delegate :institutional_LSID, to: :repository, prefix: :repository, allow_nil: true

  # Preparation delegations
  delegate :name, to: :preparation_type, prefix: :preparation_type, allow_nil: true

  has_one :accession_provider_role, class_name: 'AccessionProvider', as: :role_object, dependent: :destroy
  has_one :accession_provider, through: :accession_provider_role, source: :person
  has_one :deaccession_recipient_role, class_name: 'DeaccessionRecipient', as: :role_object, dependent: :destroy
  has_one :deaccession_recipient, through: :deaccession_recipient_role, source: :person

  # TODO: Deprecate these models.  Semantics also confuse with origin relationship.
  has_many :derived_collection_objects, inverse_of: :collection_object, dependent: :restrict_with_error
  has_many :collection_object_observations, through: :derived_collection_objects, inverse_of: :collection_objects

  has_many :sqed_depictions, through: :depictions, dependent: :restrict_with_error

  belongs_to :collecting_event, inverse_of: :collection_objects
  belongs_to :preparation_type, inverse_of: :collection_objects
  belongs_to :ranged_lot_category, inverse_of: :ranged_lots
  belongs_to :repository, inverse_of: :collection_objects
  belongs_to :current_repository, class_name: 'Repository', inverse_of: :collection_objects

  has_many :georeferences, through: :collecting_event
  has_many :geographic_items, through: :georeferences

  has_many :collectors, through: :collecting_event

  accepts_nested_attributes_for :collecting_event, allow_destroy: true, reject_if: :reject_collecting_event

  before_validation :assign_type_if_total_or_ranged_lot_category_id_provided

  validates_presence_of :type
  validate :check_that_either_total_or_ranged_lot_category_id_is_present
  validate :check_that_both_of_category_and_total_are_not_present
  validate :collecting_event_belongs_to_project

  soft_validate(
    :sv_missing_accession_fields,
    set: :missing_accession_fields,
    name: 'Missing accession fields',
    description: 'Name or Provider are not selected')

  soft_validate(
    :sv_missing_deaccession_fields,
    set: :missing_deaccession_fields,
    name: 'Missing deaccesson fields',
    description: 'Date, recipient, or reason are not specified')

  scope :with_sequence_name, ->(name) { joins(sequence_join_hack_sql).where(sequences: {name:}) }
  scope :via_descriptor, ->(descriptor) { joins(sequence_join_hack_sql).where(sequences: {id: descriptor.sequences}) }

  has_many :extracts, through: :origin_relationships, source: :new_object, source_type: 'Extract'
  has_many :sequences, through: :extracts

  # This is a hack, maybe related to a Rails 5.1 bug.
  # It returns the SQL that works in 5.0/4.2 that
  # links CollectionObject to Sequences:
  # joins(derived_extracts: [:derived_sequences])
  def self.sequence_join_hack_sql
    %Q{INNER JOIN  "origin_relationships"
               ON  "origin_relationships"."old_object_id" = "collection_objects"."id"
                  AND  "origin_relationships"."new_object_type" = 'Extract'
                  AND  "origin_relationships"."old_object_type" = 'CollectionObject'
       INNER JOIN  "extracts"
               ON  "extracts"."id" =  "origin_relationships"."new_object_id"
       INNER JOIN  "origin_relationships" "origin_relationships_extracts_join"
               ON  "origin_relationships_extracts_join"."old_object_id" = "extracts"."id"
                  AND  "origin_relationships_extracts_join"."new_object_type" = 'Sequence'
                  AND  "origin_relationships_extracts_join"."old_object_type" = 'Extract'
       INNER JOIN  "sequences"
               ON  "sequences"."id" = "origin_relationships_extracts_join"."new_object_id"}
  end

  def self.batch_update(params)
    request = QueryBatchRequest.new(
      async_cutoff: params[:async_cutoff] || 50,
      klass: 'CollectionObject',
      object_filter_params: params[:collection_object_query],
      object_params: params[:collection_object],
      preview: params[:preview],
    )

    request.cap = 1000

    query_batch_update(request)
  end

  def self.batch_update_dwc_occurrence(params)
    q = Queries::CollectionObject::Filter.new(params).all

    r = BatchResponse.new
    r.method = 'batch_update_dwc_occurrence'
    r.klass = 'CollectionObject'

    c = q.all.count

    if c == 0 || c > 10000
      r.cap_reason = 'Too many (or no) collection objects (max 10k)'
      return r
    end

    if c < 51
      q.each do |co|
        co.set_dwc_occurrence
        r.updated.push co.id
      end
    else
      r.async = true
      q.each do |co|
        co.dwc_occurrence_update_query
      end
    end

    return r
  end

  def dwc_occurrence_update_query
    self.send(:set_dwc_occurrence)
  end

  handle_asynchronously :dwc_occurrence_update_query, run_at: Proc.new { 1.second.from_now }, queue: :query_batch_update

  # TODO: move to a helper
  def self.breakdown_status(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array

    breakdown = {
      total_objects:     collection_objects.length,
      collecting_events: {},
      determinations:    {},
      bio_overview:      []
    }

    breakdown.merge!(breakdown_buffered(collection_objects))

    collection_objects.each do |co|
      breakdown[:collecting_events].merge!(co => co.collecting_event) if co.collecting_event
      breakdown[:determinations].merge!(co => co.taxon_determinations) if co.taxon_determinations.load.any?
      breakdown[:bio_overview].push([co.total, co.biocuration_classes.collect { |a| a.name }])
    end

    breakdown
  end

  # @return [Hash]
  #   a unque list of buffered_ values observed in the collection objects passed
  def self.breakdown_buffered(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array
    breakdown = {}
    categories = BUFFERED_ATTRIBUTES

    categories.each do |c|
      breakdown[c] = []
    end

    categories.each do |c|
      collection_objects.each do |co|
        breakdown[c].push co.send(c)
      end
    end

    categories.each do |c|
      breakdown[c].uniq!
    end

    breakdown
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.earliest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:end_date_year)

    return EARLIEST_DATE if a.nil? && b.nil?  # 1700-01-01

    d = nil

    if a && b
      if a < b
        d = a
      end
    else
      d = a || b
    end
    d.to_s + '-01-01'
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.latest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:end_date_year)

    c = Time.now.strftime('%Y-%m-%d')

    return c if a.nil? && b.nil?

    d = nil

    if a && b
      if a > b
        d = a
      end
    else
      d = a || b
    end

    d.to_s + '/12/31'
  end

  # TODO: Clarify this.
  # CAREFULL - this isn't _in_, this is *with*, if it was in it would be spatial query, not a join(:geographic_items)
  #
  # Find all collection objects which have collecting events which have georeferences which have geographic_items which
  # are located within the geographic item supplied
  # @param [GeographicItem] geographic_item_id
  # @return [Scope] of CollectionObject
  def self.in_geographic_item(geographic_item, limit, steps = false)
    geographic_item_id = geographic_item.id
    if steps
      gi = GeographicItem.find(geographic_item_id)
      # find the geographic_items inside gi
      step_1 = GeographicItem.is_contained_by('any', gi) # .pluck(:id)
      # find the georeferences from the geographic_items
      step_2 = step_1.map(&:georeferences).uniq.flatten
      # find the collecting events connected to the georeferences
      step_3 = step_2.map(&:collecting_event).uniq.flatten
      # find the collection objects associated with the collecting events
      step_4 = step_3.map(&:collection_objects).flatten.map(&:id).uniq
      retval = CollectionObject.where(id: step_4.sort)
    else
      retval = CollectionObject.joins(:geographic_items)
        .where(GeographicItem.contained_by_where_sql(geographic_item.id))
        .limit(limit)
        .includes(:data_attributes, :collecting_event)
    end
    retval
  end

  # TODO: deprecate
  def self.selected_column_names
    @selected_column_names = {
      ce: {in: {}, im: {}},
      co: {in: {}, im: {}},
      bc: {in: {}, im: {}}
    } if @selected_column_names.nil?
    @selected_column_names
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collecting events
  # decode which headers to be displayed for collecting events
  def self.ce_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:ce][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:ce][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.ce_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.collecting_event.internal_attributes
      all_import_das   = collection_object.collecting_event.import_attributes
      group            = collection[:ce]
      unless group.nil?
        group.each_key { |type_key|
          group[type_key.to_sym].each_key { |header|
            this_val = nil
            case type_key.to_sym
            when :in
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            when :im
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            else
            end
          }
        }
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collection objects
  # decode which headers to be displayed for collection objects
  def self.co_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:co][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:co][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.co_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.internal_attributes
      all_import_das   = collection_object.import_attributes
      group            = collection[:co]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = nil
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
        unless group.empty?
          unless group[:im].empty?
            group[:im].each_key { |header|
              this_val = nil
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for biocuration classifications
  # decode which headers to be displayed for biocuration classifications
  def self.bc_headers(project_id)
    CollectionObject.selected_column_names
    # add selectable column names (unselected) to the column name list list
    BiocurationClass.where(project_id:).map(&:name).each { |column_name|
      @selected_column_names[:bc][:in][column_name] = {checked: '0'}
    }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.bc_attributes(collection_object, col_defs)
    retval = []
    collection = col_defs
    unless collection.nil?
      group = collection[:bc]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = collection_object.biocuration_classes.map(&:name).include?(header) ? '1' : '0'
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Array] collecting_event_ids (e.g., from CollectingEvent.in_date_range)
  # @param [Array] area_object_ids (e.g., from GeographicItem.gather_selected_data())
  # @return [Scope] of intersection of collecting events (usually by date range)
  #   and collection objects (usually by inclusion in geographic areas/items)
  def self.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id)
    collecting_events_clause = {collecting_event_id: collecting_event_ids, project: project_id}
    area_objects_clause      = {id: area_object_ids, project: project_id}

    if (collecting_event_ids.empty?)
      collecting_events_clause = {project: project_id}
    end

    if (area_object_ids.empty?)
      area_objects_clause = {}
      if (area_set)
        area_objects_clause = 'false'
      end
    end

    retval = CollectionObject.joins(:collecting_event)
      .where(collecting_events_clause)
      .where(area_objects_clause)
    retval
  end

  # TODO: move to filter
  # @param [Hash] search_start_date string in form 'yyyy-mm-dd'
  # @param [Hash] search_end_date string in form 'yyyy-mm-dd'
  # @param [Hash] partial_overlap 'on' or 'off'
  # @return [Scope] of selected collection objects through collecting events with georeferences, remember to scope to project!
  def self.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on')
    allow_partial = (partial_overlap.downcase == 'off' ? false : true) # TODO: Just get the correct values from the form!
    q = Queries::CollectingEvent::Filter.new(start_date: search_start_date, end_date: search_end_date, partial_overlap_dates: allow_partial)
    joins(:collecting_event).where(q.between_date_range_facet.to_sql)
  end

  # @param used_on [String] required, one of `TaxonDetermination`, `BiologicalAssociation`
  # @return [Scope]
  #    the max 10 most recently used collection_objects, as `used_on`
  def self.used_recently(user_id, project_id, used_on = '')
    return [] if used_on != 'TaxonDetermination' && used_on != 'BiologicalAssociation'
    t = case used_on
        when 'TaxonDetermination'
          TaxonDetermination.arel_table
        when 'BiologicalAssociation'
          BiologicalAssociation.arel_table
        end

    p = CollectionObject.arel_table

    # i is a select manager
    i = case used_on
        when 'BiologicalAssociation'
          t.project(t['biological_association_subject_id'], t['updated_at']).from(t)
            .where(
              t['updated_at'].gt(1.week.ago).and(
                t['biological_association_subject_type'].eq('CollectionObject')
              )
            )
              .where(t['updated_by_id'].eq(user_id))
              .where(t['project_id'].eq(project_id))
              .order(t['updated_at'].desc)
        else
          # TODO: update to reference new TaxonDetermination
          t.project(t['taxon_determination_object_id'], t['taxon_determination_object_type'], t['updated_at']).from(t)
            .where(t['updated_at'].gt( 1.week.ago ))
            .where(t['updated_by_id'].eq(user_id))
            .where(t['project_id'].eq(project_id))
            .order(t['updated_at'].desc)
        end

    # z is a table alias
    z = i.as('recent_t')

    j = case used_on
        when 'BiologicalAssociation'
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(
            z['biological_association_subject_id'].eq(p['id'])
          ))
        else
          # TODO: needs to be fixed to scope the taxon_determination_object_type
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(z['taxon_determination_object_id'].eq(p['id'])))
        end

    CollectionObject.joins(j).pluck(:id).uniq
  end

  # @params target [String] one of `TaxonDetermination`, `BiologicalAssociation` , nil
  # @return [Hash] otus optimized for user selection
  def self.select_optimized(user_id, project_id, target = nil)
    r = used_recently(user_id, project_id, target)
    h = {
      quick: [],
      pinboard: CollectionObject.pinned_by(user_id).where(project_id:).to_a,
      recent: []
    }

    if target && !r.empty?
      n = target.tableize.to_sym
      h[:recent] = CollectionObject.where('"collection_objects"."id" IN (?)', r.first(10) ).to_a
      h[:quick] = (CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a  +
                   CollectionObject.where('"collection_objects"."id" IN (?)', r.first(4) ).to_a).uniq
    else
      h[:recent] = CollectionObject.where(project_id:, updated_by_id: user_id).order('updated_at DESC').limit(10).to_a
      h[:quick] = CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a
    end

    h
  end

  # TODO: Unify with Extract in concern
  # @return [Identifier::Local::CatalogNumber, nil]
  #   the first (position) catalog number for this collection object, either on specimen, or container
  def preferred_catalog_number
    if i = Identifier::Local::CatalogNumber.where(identifier_object: self).order(:position).first
      i
    else
      if container
        container.identifiers.where(identifiers: {type: 'Identifier::Local::CatalogNumber'}).order(:position).first
      else
        nil
      end
    end
  end

  # @return [Identifier::Local::RecordNumber, nil]
  #   the first (position) record_Number, on a specimen
  #   !1 Doesn't presently support containers
  def preferred_record_number
    Identifier::Local::RecordNumber.where(identifier_object: self).order(:position).first
  end

  def geographic_name_classification
    # don't load the whole object, just the fields we need
    if a = DwcOccurrence.where(dwc_occurrence_object: self).select(:country, :stateProvince, :county).first

      c = a.country
      s = a.stateProvince
      y = a.county

      v = ::Utilities::Geo::DICTIONARY[c]
      c = v if v
      # s = v if v = ::Utilities::Geo::DICTIONARY[s] # None in there yet
      # y = v if v = ::Utilities::Geo::DICTIONARY[y] # None in there yet

      return {
        country: c,
        state: s,
        county: y
      }
    end
  end

  # return [Boolean]
  #    True if instance is a subclass of BiologicalCollectionObject
  def is_biological?
    self.class <= BiologicalCollectionObject ? true : false
  end

  def annotations
    h = annotations_hash
    (h['biocuration classifications'] = biocuration_classes) if is_biological? && biocuration_classifications.load.any?
    h
  end

  def sv_missing_accession_fields
    soft_validations.add(:accessioned_at, 'Date is not selected') if self.accessioned_at.nil? && !self.accession_provider.nil?
    soft_validations.add(:base, 'Provider is not selected') if !self.accessioned_at.nil? && self.accession_provider.nil?
  end

  def sv_missing_deaccession_fields
    soft_validations.add(:deaccessioned_at, 'Date is not selected') if self.deaccessioned_at.nil? && self.deaccession_reason.present?
    soft_validations.add(:base, 'Recipient is not selected') if self.deaccession_recipient.nil? && self.deaccession_reason && self.deaccessioned_at
    soft_validations.add(:deaccession_reason, 'Reason is is not defined') if self.deaccession_reason.blank? && self.deaccession_recipient && self.deaccessioned_at
  end

  def sv_missing_determination
    # see biological_collection_object
  end

  def sv_missing_collecting_event
    # see biological_collection_object
  end

  def sv_missing_preparation_type
    # see biological_collection_object
  end

  def sv_missing_repository
    # WHY? -  see biological_collection_object
  end

  def sv_missing_biocuration_classification
    # see biological_collection_object
  end

  # See Depiction#destroy_image_stub_collection_object
  # Used to determin if the CO can be
  # destroy after moving an image off
  # this object.
  def is_image_stub?
    r = [
      collecting_event_id.blank?,
      !depictions.reload.any?,
      identifiers.count <= 1,
      !taxon_determinations.any?,
      !type_materials.any?,
      !citations.any?,
      !data_attributes.any?,
      !notes.any?,
      !observations.any?
    ]

   !r.include?(false)

  end

  protected

  def collecting_event_belongs_to_project
    if collecting_event&.persisted? && (Current.project_id || project_id)
      errors.add(:base, 'collecting event is not from this project') if collecting_event.project_id != (Current.project_id || project_id)
    end
  end

  def check_that_both_of_category_and_total_are_not_present
    errors.add(:ranged_lot_category_id, 'Both ranged_lot_category and total can not be set') if ranged_lot_category_id.present? && total.present?
  end

  def check_that_either_total_or_ranged_lot_category_id_is_present
    errors.add(:base, 'Either total or a ranged lot category must be provided') if ranged_lot_category_id.blank? && total.blank?
  end

  def assign_type_if_total_or_ranged_lot_category_id_provided
    if self.total == 1
      self.type = 'Specimen'
    elsif self.total.to_i > 1
      self.type = 'Lot'
    elsif total.nil? && ranged_lot_category_id.present?
      self.type = 'RangedLot'
    end
    true
  end

  def reject_collecting_event(attributed)
    reject = true
    CollectingEvent.core_attributes.each do |a|
      if attributed[a].present?
        reject = false
        break
      end
    end
    # !! does not account for georeferences_attributes!
    reject
  end

end

#preparation_type_id ⇒ Integer

How the collection object was prepared. Draws from a controlled set of values shared by all projects. For example “slide mounted”. See PreparationType.

Returns:

• (Integer)

# File 'app/models/collection_object.rb', line 64

class CollectionObject < ApplicationRecord
  include GlobalID::Identification
  include Housekeeping

  include Shared::Citations
  include Shared::Containable
  include Shared::DataAttributes
  include Shared::Loanable
  include Shared::Identifiers
  include Shared::Notes
  include Shared::Tags
  include Shared::Depictions
  include Shared::OriginRelationship
  include Shared::Confidences
  include Shared::ProtocolRelationships
  include Shared::HasPapertrail
  include Shared::Observations
  include Shared::IsData
  include Shared::QueryBatchUpdate
  include SoftValidation

  include Shared::BiologicalExtensions

  include Shared::Taxonomy # at present must be before IsDwcOccurence
  include Shared::IsDwcOccurrence
  include CollectionObject::DwcExtensions

  ignore_whitespace_on(:buffered_collecting_event, :buffered_determinations, :buffered_other_labels)

  # TODO: move to export
  CO_OTU_HEADERS = %w{OTU OTU\ name Family Genus Species Country State County Locality Latitude Longitude}.freeze

  BUFFERED_ATTRIBUTES = %i{buffered_collecting_event buffered_determinations buffered_other_labels}.freeze

  GRAPH_ENTRY_POINTS = [:biological_associations, :data_attributes, :taxon_determinations, :biocuration_classifications, :collecting_event, :origin_relationships, :extracts, :observation_matrices]

  # Identifier delegations
  # .catalog_number_cached
  delegate :cached, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true
  # .catalog_number_namespace
  delegate :namespace, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true

  # .record_number_cached
  delegate :cached, to: :preferred_record_number, prefix: :record_number, allow_nil: true
  # .record_number_namespace
  delegate :namespace, to: :preferred_record_number, prefix: :record_number, allow_nil: true

  # CollectingEvent delegations
  delegate :map_center, to: :collecting_event, prefix: :collecting_event, allow_nil: true
  delegate :collectors, to: :collecting_event, prefix: :collecting_event, allow_nil: true

  # Repository delegations
  delegate :acronym, to: :repository, prefix: :repository, allow_nil: true
  delegate :url, to: :repository, prefix: :repository, allow_nil: true
  delegate :institutional_LSID, to: :repository, prefix: :repository, allow_nil: true

  # Preparation delegations
  delegate :name, to: :preparation_type, prefix: :preparation_type, allow_nil: true

  has_one :accession_provider_role, class_name: 'AccessionProvider', as: :role_object, dependent: :destroy
  has_one :accession_provider, through: :accession_provider_role, source: :person
  has_one :deaccession_recipient_role, class_name: 'DeaccessionRecipient', as: :role_object, dependent: :destroy
  has_one :deaccession_recipient, through: :deaccession_recipient_role, source: :person

  # TODO: Deprecate these models.  Semantics also confuse with origin relationship.
  has_many :derived_collection_objects, inverse_of: :collection_object, dependent: :restrict_with_error
  has_many :collection_object_observations, through: :derived_collection_objects, inverse_of: :collection_objects

  has_many :sqed_depictions, through: :depictions, dependent: :restrict_with_error

  belongs_to :collecting_event, inverse_of: :collection_objects
  belongs_to :preparation_type, inverse_of: :collection_objects
  belongs_to :ranged_lot_category, inverse_of: :ranged_lots
  belongs_to :repository, inverse_of: :collection_objects
  belongs_to :current_repository, class_name: 'Repository', inverse_of: :collection_objects

  has_many :georeferences, through: :collecting_event
  has_many :geographic_items, through: :georeferences

  has_many :collectors, through: :collecting_event

  accepts_nested_attributes_for :collecting_event, allow_destroy: true, reject_if: :reject_collecting_event

  before_validation :assign_type_if_total_or_ranged_lot_category_id_provided

  validates_presence_of :type
  validate :check_that_either_total_or_ranged_lot_category_id_is_present
  validate :check_that_both_of_category_and_total_are_not_present
  validate :collecting_event_belongs_to_project

  soft_validate(
    :sv_missing_accession_fields,
    set: :missing_accession_fields,
    name: 'Missing accession fields',
    description: 'Name or Provider are not selected')

  soft_validate(
    :sv_missing_deaccession_fields,
    set: :missing_deaccession_fields,
    name: 'Missing deaccesson fields',
    description: 'Date, recipient, or reason are not specified')

  scope :with_sequence_name, ->(name) { joins(sequence_join_hack_sql).where(sequences: {name:}) }
  scope :via_descriptor, ->(descriptor) { joins(sequence_join_hack_sql).where(sequences: {id: descriptor.sequences}) }

  has_many :extracts, through: :origin_relationships, source: :new_object, source_type: 'Extract'
  has_many :sequences, through: :extracts

  # This is a hack, maybe related to a Rails 5.1 bug.
  # It returns the SQL that works in 5.0/4.2 that
  # links CollectionObject to Sequences:
  # joins(derived_extracts: [:derived_sequences])
  def self.sequence_join_hack_sql
    %Q{INNER JOIN  "origin_relationships"
               ON  "origin_relationships"."old_object_id" = "collection_objects"."id"
                  AND  "origin_relationships"."new_object_type" = 'Extract'
                  AND  "origin_relationships"."old_object_type" = 'CollectionObject'
       INNER JOIN  "extracts"
               ON  "extracts"."id" =  "origin_relationships"."new_object_id"
       INNER JOIN  "origin_relationships" "origin_relationships_extracts_join"
               ON  "origin_relationships_extracts_join"."old_object_id" = "extracts"."id"
                  AND  "origin_relationships_extracts_join"."new_object_type" = 'Sequence'
                  AND  "origin_relationships_extracts_join"."old_object_type" = 'Extract'
       INNER JOIN  "sequences"
               ON  "sequences"."id" = "origin_relationships_extracts_join"."new_object_id"}
  end

  def self.batch_update(params)
    request = QueryBatchRequest.new(
      async_cutoff: params[:async_cutoff] || 50,
      klass: 'CollectionObject',
      object_filter_params: params[:collection_object_query],
      object_params: params[:collection_object],
      preview: params[:preview],
    )

    request.cap = 1000

    query_batch_update(request)
  end

  def self.batch_update_dwc_occurrence(params)
    q = Queries::CollectionObject::Filter.new(params).all

    r = BatchResponse.new
    r.method = 'batch_update_dwc_occurrence'
    r.klass = 'CollectionObject'

    c = q.all.count

    if c == 0 || c > 10000
      r.cap_reason = 'Too many (or no) collection objects (max 10k)'
      return r
    end

    if c < 51
      q.each do |co|
        co.set_dwc_occurrence
        r.updated.push co.id
      end
    else
      r.async = true
      q.each do |co|
        co.dwc_occurrence_update_query
      end
    end

    return r
  end

  def dwc_occurrence_update_query
    self.send(:set_dwc_occurrence)
  end

  handle_asynchronously :dwc_occurrence_update_query, run_at: Proc.new { 1.second.from_now }, queue: :query_batch_update

  # TODO: move to a helper
  def self.breakdown_status(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array

    breakdown = {
      total_objects:     collection_objects.length,
      collecting_events: {},
      determinations:    {},
      bio_overview:      []
    }

    breakdown.merge!(breakdown_buffered(collection_objects))

    collection_objects.each do |co|
      breakdown[:collecting_events].merge!(co => co.collecting_event) if co.collecting_event
      breakdown[:determinations].merge!(co => co.taxon_determinations) if co.taxon_determinations.load.any?
      breakdown[:bio_overview].push([co.total, co.biocuration_classes.collect { |a| a.name }])
    end

    breakdown
  end

  # @return [Hash]
  #   a unque list of buffered_ values observed in the collection objects passed
  def self.breakdown_buffered(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array
    breakdown = {}
    categories = BUFFERED_ATTRIBUTES

    categories.each do |c|
      breakdown[c] = []
    end

    categories.each do |c|
      collection_objects.each do |co|
        breakdown[c].push co.send(c)
      end
    end

    categories.each do |c|
      breakdown[c].uniq!
    end

    breakdown
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.earliest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:end_date_year)

    return EARLIEST_DATE if a.nil? && b.nil?  # 1700-01-01

    d = nil

    if a && b
      if a < b
        d = a
      end
    else
      d = a || b
    end
    d.to_s + '-01-01'
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.latest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:end_date_year)

    c = Time.now.strftime('%Y-%m-%d')

    return c if a.nil? && b.nil?

    d = nil

    if a && b
      if a > b
        d = a
      end
    else
      d = a || b
    end

    d.to_s + '/12/31'
  end

  # TODO: Clarify this.
  # CAREFULL - this isn't _in_, this is *with*, if it was in it would be spatial query, not a join(:geographic_items)
  #
  # Find all collection objects which have collecting events which have georeferences which have geographic_items which
  # are located within the geographic item supplied
  # @param [GeographicItem] geographic_item_id
  # @return [Scope] of CollectionObject
  def self.in_geographic_item(geographic_item, limit, steps = false)
    geographic_item_id = geographic_item.id
    if steps
      gi = GeographicItem.find(geographic_item_id)
      # find the geographic_items inside gi
      step_1 = GeographicItem.is_contained_by('any', gi) # .pluck(:id)
      # find the georeferences from the geographic_items
      step_2 = step_1.map(&:georeferences).uniq.flatten
      # find the collecting events connected to the georeferences
      step_3 = step_2.map(&:collecting_event).uniq.flatten
      # find the collection objects associated with the collecting events
      step_4 = step_3.map(&:collection_objects).flatten.map(&:id).uniq
      retval = CollectionObject.where(id: step_4.sort)
    else
      retval = CollectionObject.joins(:geographic_items)
        .where(GeographicItem.contained_by_where_sql(geographic_item.id))
        .limit(limit)
        .includes(:data_attributes, :collecting_event)
    end
    retval
  end

  # TODO: deprecate
  def self.selected_column_names
    @selected_column_names = {
      ce: {in: {}, im: {}},
      co: {in: {}, im: {}},
      bc: {in: {}, im: {}}
    } if @selected_column_names.nil?
    @selected_column_names
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collecting events
  # decode which headers to be displayed for collecting events
  def self.ce_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:ce][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:ce][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.ce_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.collecting_event.internal_attributes
      all_import_das   = collection_object.collecting_event.import_attributes
      group            = collection[:ce]
      unless group.nil?
        group.each_key { |type_key|
          group[type_key.to_sym].each_key { |header|
            this_val = nil
            case type_key.to_sym
            when :in
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            when :im
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            else
            end
          }
        }
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collection objects
  # decode which headers to be displayed for collection objects
  def self.co_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:co][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:co][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.co_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.internal_attributes
      all_import_das   = collection_object.import_attributes
      group            = collection[:co]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = nil
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
        unless group.empty?
          unless group[:im].empty?
            group[:im].each_key { |header|
              this_val = nil
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for biocuration classifications
  # decode which headers to be displayed for biocuration classifications
  def self.bc_headers(project_id)
    CollectionObject.selected_column_names
    # add selectable column names (unselected) to the column name list list
    BiocurationClass.where(project_id:).map(&:name).each { |column_name|
      @selected_column_names[:bc][:in][column_name] = {checked: '0'}
    }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.bc_attributes(collection_object, col_defs)
    retval = []
    collection = col_defs
    unless collection.nil?
      group = collection[:bc]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = collection_object.biocuration_classes.map(&:name).include?(header) ? '1' : '0'
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Array] collecting_event_ids (e.g., from CollectingEvent.in_date_range)
  # @param [Array] area_object_ids (e.g., from GeographicItem.gather_selected_data())
  # @return [Scope] of intersection of collecting events (usually by date range)
  #   and collection objects (usually by inclusion in geographic areas/items)
  def self.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id)
    collecting_events_clause = {collecting_event_id: collecting_event_ids, project: project_id}
    area_objects_clause      = {id: area_object_ids, project: project_id}

    if (collecting_event_ids.empty?)
      collecting_events_clause = {project: project_id}
    end

    if (area_object_ids.empty?)
      area_objects_clause = {}
      if (area_set)
        area_objects_clause = 'false'
      end
    end

    retval = CollectionObject.joins(:collecting_event)
      .where(collecting_events_clause)
      .where(area_objects_clause)
    retval
  end

  # TODO: move to filter
  # @param [Hash] search_start_date string in form 'yyyy-mm-dd'
  # @param [Hash] search_end_date string in form 'yyyy-mm-dd'
  # @param [Hash] partial_overlap 'on' or 'off'
  # @return [Scope] of selected collection objects through collecting events with georeferences, remember to scope to project!
  def self.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on')
    allow_partial = (partial_overlap.downcase == 'off' ? false : true) # TODO: Just get the correct values from the form!
    q = Queries::CollectingEvent::Filter.new(start_date: search_start_date, end_date: search_end_date, partial_overlap_dates: allow_partial)
    joins(:collecting_event).where(q.between_date_range_facet.to_sql)
  end

  # @param used_on [String] required, one of `TaxonDetermination`, `BiologicalAssociation`
  # @return [Scope]
  #    the max 10 most recently used collection_objects, as `used_on`
  def self.used_recently(user_id, project_id, used_on = '')
    return [] if used_on != 'TaxonDetermination' && used_on != 'BiologicalAssociation'
    t = case used_on
        when 'TaxonDetermination'
          TaxonDetermination.arel_table
        when 'BiologicalAssociation'
          BiologicalAssociation.arel_table
        end

    p = CollectionObject.arel_table

    # i is a select manager
    i = case used_on
        when 'BiologicalAssociation'
          t.project(t['biological_association_subject_id'], t['updated_at']).from(t)
            .where(
              t['updated_at'].gt(1.week.ago).and(
                t['biological_association_subject_type'].eq('CollectionObject')
              )
            )
              .where(t['updated_by_id'].eq(user_id))
              .where(t['project_id'].eq(project_id))
              .order(t['updated_at'].desc)
        else
          # TODO: update to reference new TaxonDetermination
          t.project(t['taxon_determination_object_id'], t['taxon_determination_object_type'], t['updated_at']).from(t)
            .where(t['updated_at'].gt( 1.week.ago ))
            .where(t['updated_by_id'].eq(user_id))
            .where(t['project_id'].eq(project_id))
            .order(t['updated_at'].desc)
        end

    # z is a table alias
    z = i.as('recent_t')

    j = case used_on
        when 'BiologicalAssociation'
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(
            z['biological_association_subject_id'].eq(p['id'])
          ))
        else
          # TODO: needs to be fixed to scope the taxon_determination_object_type
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(z['taxon_determination_object_id'].eq(p['id'])))
        end

    CollectionObject.joins(j).pluck(:id).uniq
  end

  # @params target [String] one of `TaxonDetermination`, `BiologicalAssociation` , nil
  # @return [Hash] otus optimized for user selection
  def self.select_optimized(user_id, project_id, target = nil)
    r = used_recently(user_id, project_id, target)
    h = {
      quick: [],
      pinboard: CollectionObject.pinned_by(user_id).where(project_id:).to_a,
      recent: []
    }

    if target && !r.empty?
      n = target.tableize.to_sym
      h[:recent] = CollectionObject.where('"collection_objects"."id" IN (?)', r.first(10) ).to_a
      h[:quick] = (CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a  +
                   CollectionObject.where('"collection_objects"."id" IN (?)', r.first(4) ).to_a).uniq
    else
      h[:recent] = CollectionObject.where(project_id:, updated_by_id: user_id).order('updated_at DESC').limit(10).to_a
      h[:quick] = CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a
    end

    h
  end

  # TODO: Unify with Extract in concern
  # @return [Identifier::Local::CatalogNumber, nil]
  #   the first (position) catalog number for this collection object, either on specimen, or container
  def preferred_catalog_number
    if i = Identifier::Local::CatalogNumber.where(identifier_object: self).order(:position).first
      i
    else
      if container
        container.identifiers.where(identifiers: {type: 'Identifier::Local::CatalogNumber'}).order(:position).first
      else
        nil
      end
    end
  end

  # @return [Identifier::Local::RecordNumber, nil]
  #   the first (position) record_Number, on a specimen
  #   !1 Doesn't presently support containers
  def preferred_record_number
    Identifier::Local::RecordNumber.where(identifier_object: self).order(:position).first
  end

  def geographic_name_classification
    # don't load the whole object, just the fields we need
    if a = DwcOccurrence.where(dwc_occurrence_object: self).select(:country, :stateProvince, :county).first

      c = a.country
      s = a.stateProvince
      y = a.county

      v = ::Utilities::Geo::DICTIONARY[c]
      c = v if v
      # s = v if v = ::Utilities::Geo::DICTIONARY[s] # None in there yet
      # y = v if v = ::Utilities::Geo::DICTIONARY[y] # None in there yet

      return {
        country: c,
        state: s,
        county: y
      }
    end
  end

  # return [Boolean]
  #    True if instance is a subclass of BiologicalCollectionObject
  def is_biological?
    self.class <= BiologicalCollectionObject ? true : false
  end

  def annotations
    h = annotations_hash
    (h['biocuration classifications'] = biocuration_classes) if is_biological? && biocuration_classifications.load.any?
    h
  end

  def sv_missing_accession_fields
    soft_validations.add(:accessioned_at, 'Date is not selected') if self.accessioned_at.nil? && !self.accession_provider.nil?
    soft_validations.add(:base, 'Provider is not selected') if !self.accessioned_at.nil? && self.accession_provider.nil?
  end

  def sv_missing_deaccession_fields
    soft_validations.add(:deaccessioned_at, 'Date is not selected') if self.deaccessioned_at.nil? && self.deaccession_reason.present?
    soft_validations.add(:base, 'Recipient is not selected') if self.deaccession_recipient.nil? && self.deaccession_reason && self.deaccessioned_at
    soft_validations.add(:deaccession_reason, 'Reason is is not defined') if self.deaccession_reason.blank? && self.deaccession_recipient && self.deaccessioned_at
  end

  def sv_missing_determination
    # see biological_collection_object
  end

  def sv_missing_collecting_event
    # see biological_collection_object
  end

  def sv_missing_preparation_type
    # see biological_collection_object
  end

  def sv_missing_repository
    # WHY? -  see biological_collection_object
  end

  def sv_missing_biocuration_classification
    # see biological_collection_object
  end

  # See Depiction#destroy_image_stub_collection_object
  # Used to determin if the CO can be
  # destroy after moving an image off
  # this object.
  def is_image_stub?
    r = [
      collecting_event_id.blank?,
      !depictions.reload.any?,
      identifiers.count <= 1,
      !taxon_determinations.any?,
      !type_materials.any?,
      !citations.any?,
      !data_attributes.any?,
      !notes.any?,
      !observations.any?
    ]

   !r.include?(false)

  end

  protected

  def collecting_event_belongs_to_project
    if collecting_event&.persisted? && (Current.project_id || project_id)
      errors.add(:base, 'collecting event is not from this project') if collecting_event.project_id != (Current.project_id || project_id)
    end
  end

  def check_that_both_of_category_and_total_are_not_present
    errors.add(:ranged_lot_category_id, 'Both ranged_lot_category and total can not be set') if ranged_lot_category_id.present? && total.present?
  end

  def check_that_either_total_or_ranged_lot_category_id_is_present
    errors.add(:base, 'Either total or a ranged lot category must be provided') if ranged_lot_category_id.blank? && total.blank?
  end

  def assign_type_if_total_or_ranged_lot_category_id_provided
    if self.total == 1
      self.type = 'Specimen'
    elsif self.total.to_i > 1
      self.type = 'Lot'
    elsif total.nil? && ranged_lot_category_id.present?
      self.type = 'RangedLot'
    end
    true
  end

  def reject_collecting_event(attributed)
    reject = true
    CollectingEvent.core_attributes.each do |a|
      if attributed[a].present?
        reject = false
        break
      end
    end
    # !! does not account for georeferences_attributes!
    reject
  end

end

#project_id ⇒ Integer

the project ID

Returns:

• (Integer)

# File 'app/models/collection_object.rb', line 64

class CollectionObject < ApplicationRecord
  include GlobalID::Identification
  include Housekeeping

  include Shared::Citations
  include Shared::Containable
  include Shared::DataAttributes
  include Shared::Loanable
  include Shared::Identifiers
  include Shared::Notes
  include Shared::Tags
  include Shared::Depictions
  include Shared::OriginRelationship
  include Shared::Confidences
  include Shared::ProtocolRelationships
  include Shared::HasPapertrail
  include Shared::Observations
  include Shared::IsData
  include Shared::QueryBatchUpdate
  include SoftValidation

  include Shared::BiologicalExtensions

  include Shared::Taxonomy # at present must be before IsDwcOccurence
  include Shared::IsDwcOccurrence
  include CollectionObject::DwcExtensions

  ignore_whitespace_on(:buffered_collecting_event, :buffered_determinations, :buffered_other_labels)

  # TODO: move to export
  CO_OTU_HEADERS = %w{OTU OTU\ name Family Genus Species Country State County Locality Latitude Longitude}.freeze

  BUFFERED_ATTRIBUTES = %i{buffered_collecting_event buffered_determinations buffered_other_labels}.freeze

  GRAPH_ENTRY_POINTS = [:biological_associations, :data_attributes, :taxon_determinations, :biocuration_classifications, :collecting_event, :origin_relationships, :extracts, :observation_matrices]

  # Identifier delegations
  # .catalog_number_cached
  delegate :cached, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true
  # .catalog_number_namespace
  delegate :namespace, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true

  # .record_number_cached
  delegate :cached, to: :preferred_record_number, prefix: :record_number, allow_nil: true
  # .record_number_namespace
  delegate :namespace, to: :preferred_record_number, prefix: :record_number, allow_nil: true

  # CollectingEvent delegations
  delegate :map_center, to: :collecting_event, prefix: :collecting_event, allow_nil: true
  delegate :collectors, to: :collecting_event, prefix: :collecting_event, allow_nil: true

  # Repository delegations
  delegate :acronym, to: :repository, prefix: :repository, allow_nil: true
  delegate :url, to: :repository, prefix: :repository, allow_nil: true
  delegate :institutional_LSID, to: :repository, prefix: :repository, allow_nil: true

  # Preparation delegations
  delegate :name, to: :preparation_type, prefix: :preparation_type, allow_nil: true

  has_one :accession_provider_role, class_name: 'AccessionProvider', as: :role_object, dependent: :destroy
  has_one :accession_provider, through: :accession_provider_role, source: :person
  has_one :deaccession_recipient_role, class_name: 'DeaccessionRecipient', as: :role_object, dependent: :destroy
  has_one :deaccession_recipient, through: :deaccession_recipient_role, source: :person

  # TODO: Deprecate these models.  Semantics also confuse with origin relationship.
  has_many :derived_collection_objects, inverse_of: :collection_object, dependent: :restrict_with_error
  has_many :collection_object_observations, through: :derived_collection_objects, inverse_of: :collection_objects

  has_many :sqed_depictions, through: :depictions, dependent: :restrict_with_error

  belongs_to :collecting_event, inverse_of: :collection_objects
  belongs_to :preparation_type, inverse_of: :collection_objects
  belongs_to :ranged_lot_category, inverse_of: :ranged_lots
  belongs_to :repository, inverse_of: :collection_objects
  belongs_to :current_repository, class_name: 'Repository', inverse_of: :collection_objects

  has_many :georeferences, through: :collecting_event
  has_many :geographic_items, through: :georeferences

  has_many :collectors, through: :collecting_event

  accepts_nested_attributes_for :collecting_event, allow_destroy: true, reject_if: :reject_collecting_event

  before_validation :assign_type_if_total_or_ranged_lot_category_id_provided

  validates_presence_of :type
  validate :check_that_either_total_or_ranged_lot_category_id_is_present
  validate :check_that_both_of_category_and_total_are_not_present
  validate :collecting_event_belongs_to_project

  soft_validate(
    :sv_missing_accession_fields,
    set: :missing_accession_fields,
    name: 'Missing accession fields',
    description: 'Name or Provider are not selected')

  soft_validate(
    :sv_missing_deaccession_fields,
    set: :missing_deaccession_fields,
    name: 'Missing deaccesson fields',
    description: 'Date, recipient, or reason are not specified')

  scope :with_sequence_name, ->(name) { joins(sequence_join_hack_sql).where(sequences: {name:}) }
  scope :via_descriptor, ->(descriptor) { joins(sequence_join_hack_sql).where(sequences: {id: descriptor.sequences}) }

  has_many :extracts, through: :origin_relationships, source: :new_object, source_type: 'Extract'
  has_many :sequences, through: :extracts

  # This is a hack, maybe related to a Rails 5.1 bug.
  # It returns the SQL that works in 5.0/4.2 that
  # links CollectionObject to Sequences:
  # joins(derived_extracts: [:derived_sequences])
  def self.sequence_join_hack_sql
    %Q{INNER JOIN  "origin_relationships"
               ON  "origin_relationships"."old_object_id" = "collection_objects"."id"
                  AND  "origin_relationships"."new_object_type" = 'Extract'
                  AND  "origin_relationships"."old_object_type" = 'CollectionObject'
       INNER JOIN  "extracts"
               ON  "extracts"."id" =  "origin_relationships"."new_object_id"
       INNER JOIN  "origin_relationships" "origin_relationships_extracts_join"
               ON  "origin_relationships_extracts_join"."old_object_id" = "extracts"."id"
                  AND  "origin_relationships_extracts_join"."new_object_type" = 'Sequence'
                  AND  "origin_relationships_extracts_join"."old_object_type" = 'Extract'
       INNER JOIN  "sequences"
               ON  "sequences"."id" = "origin_relationships_extracts_join"."new_object_id"}
  end

  def self.batch_update(params)
    request = QueryBatchRequest.new(
      async_cutoff: params[:async_cutoff] || 50,
      klass: 'CollectionObject',
      object_filter_params: params[:collection_object_query],
      object_params: params[:collection_object],
      preview: params[:preview],
    )

    request.cap = 1000

    query_batch_update(request)
  end

  def self.batch_update_dwc_occurrence(params)
    q = Queries::CollectionObject::Filter.new(params).all

    r = BatchResponse.new
    r.method = 'batch_update_dwc_occurrence'
    r.klass = 'CollectionObject'

    c = q.all.count

    if c == 0 || c > 10000
      r.cap_reason = 'Too many (or no) collection objects (max 10k)'
      return r
    end

    if c < 51
      q.each do |co|
        co.set_dwc_occurrence
        r.updated.push co.id
      end
    else
      r.async = true
      q.each do |co|
        co.dwc_occurrence_update_query
      end
    end

    return r
  end

  def dwc_occurrence_update_query
    self.send(:set_dwc_occurrence)
  end

  handle_asynchronously :dwc_occurrence_update_query, run_at: Proc.new { 1.second.from_now }, queue: :query_batch_update

  # TODO: move to a helper
  def self.breakdown_status(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array

    breakdown = {
      total_objects:     collection_objects.length,
      collecting_events: {},
      determinations:    {},
      bio_overview:      []
    }

    breakdown.merge!(breakdown_buffered(collection_objects))

    collection_objects.each do |co|
      breakdown[:collecting_events].merge!(co => co.collecting_event) if co.collecting_event
      breakdown[:determinations].merge!(co => co.taxon_determinations) if co.taxon_determinations.load.any?
      breakdown[:bio_overview].push([co.total, co.biocuration_classes.collect { |a| a.name }])
    end

    breakdown
  end

  # @return [Hash]
  #   a unque list of buffered_ values observed in the collection objects passed
  def self.breakdown_buffered(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array
    breakdown = {}
    categories = BUFFERED_ATTRIBUTES

    categories.each do |c|
      breakdown[c] = []
    end

    categories.each do |c|
      collection_objects.each do |co|
        breakdown[c].push co.send(c)
      end
    end

    categories.each do |c|
      breakdown[c].uniq!
    end

    breakdown
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.earliest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:end_date_year)

    return EARLIEST_DATE if a.nil? && b.nil?  # 1700-01-01

    d = nil

    if a && b
      if a < b
        d = a
      end
    else
      d = a || b
    end
    d.to_s + '-01-01'
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.latest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:end_date_year)

    c = Time.now.strftime('%Y-%m-%d')

    return c if a.nil? && b.nil?

    d = nil

    if a && b
      if a > b
        d = a
      end
    else
      d = a || b
    end

    d.to_s + '/12/31'
  end

  # TODO: Clarify this.
  # CAREFULL - this isn't _in_, this is *with*, if it was in it would be spatial query, not a join(:geographic_items)
  #
  # Find all collection objects which have collecting events which have georeferences which have geographic_items which
  # are located within the geographic item supplied
  # @param [GeographicItem] geographic_item_id
  # @return [Scope] of CollectionObject
  def self.in_geographic_item(geographic_item, limit, steps = false)
    geographic_item_id = geographic_item.id
    if steps
      gi = GeographicItem.find(geographic_item_id)
      # find the geographic_items inside gi
      step_1 = GeographicItem.is_contained_by('any', gi) # .pluck(:id)
      # find the georeferences from the geographic_items
      step_2 = step_1.map(&:georeferences).uniq.flatten
      # find the collecting events connected to the georeferences
      step_3 = step_2.map(&:collecting_event).uniq.flatten
      # find the collection objects associated with the collecting events
      step_4 = step_3.map(&:collection_objects).flatten.map(&:id).uniq
      retval = CollectionObject.where(id: step_4.sort)
    else
      retval = CollectionObject.joins(:geographic_items)
        .where(GeographicItem.contained_by_where_sql(geographic_item.id))
        .limit(limit)
        .includes(:data_attributes, :collecting_event)
    end
    retval
  end

  # TODO: deprecate
  def self.selected_column_names
    @selected_column_names = {
      ce: {in: {}, im: {}},
      co: {in: {}, im: {}},
      bc: {in: {}, im: {}}
    } if @selected_column_names.nil?
    @selected_column_names
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collecting events
  # decode which headers to be displayed for collecting events
  def self.ce_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:ce][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:ce][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.ce_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.collecting_event.internal_attributes
      all_import_das   = collection_object.collecting_event.import_attributes
      group            = collection[:ce]
      unless group.nil?
        group.each_key { |type_key|
          group[type_key.to_sym].each_key { |header|
            this_val = nil
            case type_key.to_sym
            when :in
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            when :im
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            else
            end
          }
        }
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collection objects
  # decode which headers to be displayed for collection objects
  def self.co_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:co][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:co][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.co_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.internal_attributes
      all_import_das   = collection_object.import_attributes
      group            = collection[:co]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = nil
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
        unless group.empty?
          unless group[:im].empty?
            group[:im].each_key { |header|
              this_val = nil
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for biocuration classifications
  # decode which headers to be displayed for biocuration classifications
  def self.bc_headers(project_id)
    CollectionObject.selected_column_names
    # add selectable column names (unselected) to the column name list list
    BiocurationClass.where(project_id:).map(&:name).each { |column_name|
      @selected_column_names[:bc][:in][column_name] = {checked: '0'}
    }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.bc_attributes(collection_object, col_defs)
    retval = []
    collection = col_defs
    unless collection.nil?
      group = collection[:bc]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = collection_object.biocuration_classes.map(&:name).include?(header) ? '1' : '0'
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Array] collecting_event_ids (e.g., from CollectingEvent.in_date_range)
  # @param [Array] area_object_ids (e.g., from GeographicItem.gather_selected_data())
  # @return [Scope] of intersection of collecting events (usually by date range)
  #   and collection objects (usually by inclusion in geographic areas/items)
  def self.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id)
    collecting_events_clause = {collecting_event_id: collecting_event_ids, project: project_id}
    area_objects_clause      = {id: area_object_ids, project: project_id}

    if (collecting_event_ids.empty?)
      collecting_events_clause = {project: project_id}
    end

    if (area_object_ids.empty?)
      area_objects_clause = {}
      if (area_set)
        area_objects_clause = 'false'
      end
    end

    retval = CollectionObject.joins(:collecting_event)
      .where(collecting_events_clause)
      .where(area_objects_clause)
    retval
  end

  # TODO: move to filter
  # @param [Hash] search_start_date string in form 'yyyy-mm-dd'
  # @param [Hash] search_end_date string in form 'yyyy-mm-dd'
  # @param [Hash] partial_overlap 'on' or 'off'
  # @return [Scope] of selected collection objects through collecting events with georeferences, remember to scope to project!
  def self.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on')
    allow_partial = (partial_overlap.downcase == 'off' ? false : true) # TODO: Just get the correct values from the form!
    q = Queries::CollectingEvent::Filter.new(start_date: search_start_date, end_date: search_end_date, partial_overlap_dates: allow_partial)
    joins(:collecting_event).where(q.between_date_range_facet.to_sql)
  end

  # @param used_on [String] required, one of `TaxonDetermination`, `BiologicalAssociation`
  # @return [Scope]
  #    the max 10 most recently used collection_objects, as `used_on`
  def self.used_recently(user_id, project_id, used_on = '')
    return [] if used_on != 'TaxonDetermination' && used_on != 'BiologicalAssociation'
    t = case used_on
        when 'TaxonDetermination'
          TaxonDetermination.arel_table
        when 'BiologicalAssociation'
          BiologicalAssociation.arel_table
        end

    p = CollectionObject.arel_table

    # i is a select manager
    i = case used_on
        when 'BiologicalAssociation'
          t.project(t['biological_association_subject_id'], t['updated_at']).from(t)
            .where(
              t['updated_at'].gt(1.week.ago).and(
                t['biological_association_subject_type'].eq('CollectionObject')
              )
            )
              .where(t['updated_by_id'].eq(user_id))
              .where(t['project_id'].eq(project_id))
              .order(t['updated_at'].desc)
        else
          # TODO: update to reference new TaxonDetermination
          t.project(t['taxon_determination_object_id'], t['taxon_determination_object_type'], t['updated_at']).from(t)
            .where(t['updated_at'].gt( 1.week.ago ))
            .where(t['updated_by_id'].eq(user_id))
            .where(t['project_id'].eq(project_id))
            .order(t['updated_at'].desc)
        end

    # z is a table alias
    z = i.as('recent_t')

    j = case used_on
        when 'BiologicalAssociation'
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(
            z['biological_association_subject_id'].eq(p['id'])
          ))
        else
          # TODO: needs to be fixed to scope the taxon_determination_object_type
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(z['taxon_determination_object_id'].eq(p['id'])))
        end

    CollectionObject.joins(j).pluck(:id).uniq
  end

  # @params target [String] one of `TaxonDetermination`, `BiologicalAssociation` , nil
  # @return [Hash] otus optimized for user selection
  def self.select_optimized(user_id, project_id, target = nil)
    r = used_recently(user_id, project_id, target)
    h = {
      quick: [],
      pinboard: CollectionObject.pinned_by(user_id).where(project_id:).to_a,
      recent: []
    }

    if target && !r.empty?
      n = target.tableize.to_sym
      h[:recent] = CollectionObject.where('"collection_objects"."id" IN (?)', r.first(10) ).to_a
      h[:quick] = (CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a  +
                   CollectionObject.where('"collection_objects"."id" IN (?)', r.first(4) ).to_a).uniq
    else
      h[:recent] = CollectionObject.where(project_id:, updated_by_id: user_id).order('updated_at DESC').limit(10).to_a
      h[:quick] = CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a
    end

    h
  end

  # TODO: Unify with Extract in concern
  # @return [Identifier::Local::CatalogNumber, nil]
  #   the first (position) catalog number for this collection object, either on specimen, or container
  def preferred_catalog_number
    if i = Identifier::Local::CatalogNumber.where(identifier_object: self).order(:position).first
      i
    else
      if container
        container.identifiers.where(identifiers: {type: 'Identifier::Local::CatalogNumber'}).order(:position).first
      else
        nil
      end
    end
  end

  # @return [Identifier::Local::RecordNumber, nil]
  #   the first (position) record_Number, on a specimen
  #   !1 Doesn't presently support containers
  def preferred_record_number
    Identifier::Local::RecordNumber.where(identifier_object: self).order(:position).first
  end

  def geographic_name_classification
    # don't load the whole object, just the fields we need
    if a = DwcOccurrence.where(dwc_occurrence_object: self).select(:country, :stateProvince, :county).first

      c = a.country
      s = a.stateProvince
      y = a.county

      v = ::Utilities::Geo::DICTIONARY[c]
      c = v if v
      # s = v if v = ::Utilities::Geo::DICTIONARY[s] # None in there yet
      # y = v if v = ::Utilities::Geo::DICTIONARY[y] # None in there yet

      return {
        country: c,
        state: s,
        county: y
      }
    end
  end

  # return [Boolean]
  #    True if instance is a subclass of BiologicalCollectionObject
  def is_biological?
    self.class <= BiologicalCollectionObject ? true : false
  end

  def annotations
    h = annotations_hash
    (h['biocuration classifications'] = biocuration_classes) if is_biological? && biocuration_classifications.load.any?
    h
  end

  def sv_missing_accession_fields
    soft_validations.add(:accessioned_at, 'Date is not selected') if self.accessioned_at.nil? && !self.accession_provider.nil?
    soft_validations.add(:base, 'Provider is not selected') if !self.accessioned_at.nil? && self.accession_provider.nil?
  end

  def sv_missing_deaccession_fields
    soft_validations.add(:deaccessioned_at, 'Date is not selected') if self.deaccessioned_at.nil? && self.deaccession_reason.present?
    soft_validations.add(:base, 'Recipient is not selected') if self.deaccession_recipient.nil? && self.deaccession_reason && self.deaccessioned_at
    soft_validations.add(:deaccession_reason, 'Reason is is not defined') if self.deaccession_reason.blank? && self.deaccession_recipient && self.deaccessioned_at
  end

  def sv_missing_determination
    # see biological_collection_object
  end

  def sv_missing_collecting_event
    # see biological_collection_object
  end

  def sv_missing_preparation_type
    # see biological_collection_object
  end

  def sv_missing_repository
    # WHY? -  see biological_collection_object
  end

  def sv_missing_biocuration_classification
    # see biological_collection_object
  end

  # See Depiction#destroy_image_stub_collection_object
  # Used to determin if the CO can be
  # destroy after moving an image off
  # this object.
  def is_image_stub?
    r = [
      collecting_event_id.blank?,
      !depictions.reload.any?,
      identifiers.count <= 1,
      !taxon_determinations.any?,
      !type_materials.any?,
      !citations.any?,
      !data_attributes.any?,
      !notes.any?,
      !observations.any?
    ]

   !r.include?(false)

  end

  protected

  def collecting_event_belongs_to_project
    if collecting_event&.persisted? && (Current.project_id || project_id)
      errors.add(:base, 'collecting event is not from this project') if collecting_event.project_id != (Current.project_id || project_id)
    end
  end

  def check_that_both_of_category_and_total_are_not_present
    errors.add(:ranged_lot_category_id, 'Both ranged_lot_category and total can not be set') if ranged_lot_category_id.present? && total.present?
  end

  def check_that_either_total_or_ranged_lot_category_id_is_present
    errors.add(:base, 'Either total or a ranged lot category must be provided') if ranged_lot_category_id.blank? && total.blank?
  end

  def assign_type_if_total_or_ranged_lot_category_id_provided
    if self.total == 1
      self.type = 'Specimen'
    elsif self.total.to_i > 1
      self.type = 'Lot'
    elsif total.nil? && ranged_lot_category_id.present?
      self.type = 'RangedLot'
    end
    true
  end

  def reject_collecting_event(attributed)
    reject = true
    CollectingEvent.core_attributes.each do |a|
      if attributed[a].present?
        reject = false
        break
      end
    end
    # !! does not account for georeferences_attributes!
    reject
  end

end

#ranged_lot_category_id ⇒ Integer

The id of the user-defined ranged lot category. See RangedLotCategory. When present the subclass is “RangedLot”.

Returns:

• (Integer)

# File 'app/models/collection_object.rb', line 64

class CollectionObject < ApplicationRecord
  include GlobalID::Identification
  include Housekeeping

  include Shared::Citations
  include Shared::Containable
  include Shared::DataAttributes
  include Shared::Loanable
  include Shared::Identifiers
  include Shared::Notes
  include Shared::Tags
  include Shared::Depictions
  include Shared::OriginRelationship
  include Shared::Confidences
  include Shared::ProtocolRelationships
  include Shared::HasPapertrail
  include Shared::Observations
  include Shared::IsData
  include Shared::QueryBatchUpdate
  include SoftValidation

  include Shared::BiologicalExtensions

  include Shared::Taxonomy # at present must be before IsDwcOccurence
  include Shared::IsDwcOccurrence
  include CollectionObject::DwcExtensions

  ignore_whitespace_on(:buffered_collecting_event, :buffered_determinations, :buffered_other_labels)

  # TODO: move to export
  CO_OTU_HEADERS = %w{OTU OTU\ name Family Genus Species Country State County Locality Latitude Longitude}.freeze

  BUFFERED_ATTRIBUTES = %i{buffered_collecting_event buffered_determinations buffered_other_labels}.freeze

  GRAPH_ENTRY_POINTS = [:biological_associations, :data_attributes, :taxon_determinations, :biocuration_classifications, :collecting_event, :origin_relationships, :extracts, :observation_matrices]

  # Identifier delegations
  # .catalog_number_cached
  delegate :cached, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true
  # .catalog_number_namespace
  delegate :namespace, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true

  # .record_number_cached
  delegate :cached, to: :preferred_record_number, prefix: :record_number, allow_nil: true
  # .record_number_namespace
  delegate :namespace, to: :preferred_record_number, prefix: :record_number, allow_nil: true

  # CollectingEvent delegations
  delegate :map_center, to: :collecting_event, prefix: :collecting_event, allow_nil: true
  delegate :collectors, to: :collecting_event, prefix: :collecting_event, allow_nil: true

  # Repository delegations
  delegate :acronym, to: :repository, prefix: :repository, allow_nil: true
  delegate :url, to: :repository, prefix: :repository, allow_nil: true
  delegate :institutional_LSID, to: :repository, prefix: :repository, allow_nil: true

  # Preparation delegations
  delegate :name, to: :preparation_type, prefix: :preparation_type, allow_nil: true

  has_one :accession_provider_role, class_name: 'AccessionProvider', as: :role_object, dependent: :destroy
  has_one :accession_provider, through: :accession_provider_role, source: :person
  has_one :deaccession_recipient_role, class_name: 'DeaccessionRecipient', as: :role_object, dependent: :destroy
  has_one :deaccession_recipient, through: :deaccession_recipient_role, source: :person

  # TODO: Deprecate these models.  Semantics also confuse with origin relationship.
  has_many :derived_collection_objects, inverse_of: :collection_object, dependent: :restrict_with_error
  has_many :collection_object_observations, through: :derived_collection_objects, inverse_of: :collection_objects

  has_many :sqed_depictions, through: :depictions, dependent: :restrict_with_error

  belongs_to :collecting_event, inverse_of: :collection_objects
  belongs_to :preparation_type, inverse_of: :collection_objects
  belongs_to :ranged_lot_category, inverse_of: :ranged_lots
  belongs_to :repository, inverse_of: :collection_objects
  belongs_to :current_repository, class_name: 'Repository', inverse_of: :collection_objects

  has_many :georeferences, through: :collecting_event
  has_many :geographic_items, through: :georeferences

  has_many :collectors, through: :collecting_event

  accepts_nested_attributes_for :collecting_event, allow_destroy: true, reject_if: :reject_collecting_event

  before_validation :assign_type_if_total_or_ranged_lot_category_id_provided

  validates_presence_of :type
  validate :check_that_either_total_or_ranged_lot_category_id_is_present
  validate :check_that_both_of_category_and_total_are_not_present
  validate :collecting_event_belongs_to_project

  soft_validate(
    :sv_missing_accession_fields,
    set: :missing_accession_fields,
    name: 'Missing accession fields',
    description: 'Name or Provider are not selected')

  soft_validate(
    :sv_missing_deaccession_fields,
    set: :missing_deaccession_fields,
    name: 'Missing deaccesson fields',
    description: 'Date, recipient, or reason are not specified')

  scope :with_sequence_name, ->(name) { joins(sequence_join_hack_sql).where(sequences: {name:}) }
  scope :via_descriptor, ->(descriptor) { joins(sequence_join_hack_sql).where(sequences: {id: descriptor.sequences}) }

  has_many :extracts, through: :origin_relationships, source: :new_object, source_type: 'Extract'
  has_many :sequences, through: :extracts

  # This is a hack, maybe related to a Rails 5.1 bug.
  # It returns the SQL that works in 5.0/4.2 that
  # links CollectionObject to Sequences:
  # joins(derived_extracts: [:derived_sequences])
  def self.sequence_join_hack_sql
    %Q{INNER JOIN  "origin_relationships"
               ON  "origin_relationships"."old_object_id" = "collection_objects"."id"
                  AND  "origin_relationships"."new_object_type" = 'Extract'
                  AND  "origin_relationships"."old_object_type" = 'CollectionObject'
       INNER JOIN  "extracts"
               ON  "extracts"."id" =  "origin_relationships"."new_object_id"
       INNER JOIN  "origin_relationships" "origin_relationships_extracts_join"
               ON  "origin_relationships_extracts_join"."old_object_id" = "extracts"."id"
                  AND  "origin_relationships_extracts_join"."new_object_type" = 'Sequence'
                  AND  "origin_relationships_extracts_join"."old_object_type" = 'Extract'
       INNER JOIN  "sequences"
               ON  "sequences"."id" = "origin_relationships_extracts_join"."new_object_id"}
  end

  def self.batch_update(params)
    request = QueryBatchRequest.new(
      async_cutoff: params[:async_cutoff] || 50,
      klass: 'CollectionObject',
      object_filter_params: params[:collection_object_query],
      object_params: params[:collection_object],
      preview: params[:preview],
    )

    request.cap = 1000

    query_batch_update(request)
  end

  def self.batch_update_dwc_occurrence(params)
    q = Queries::CollectionObject::Filter.new(params).all

    r = BatchResponse.new
    r.method = 'batch_update_dwc_occurrence'
    r.klass = 'CollectionObject'

    c = q.all.count

    if c == 0 || c > 10000
      r.cap_reason = 'Too many (or no) collection objects (max 10k)'
      return r
    end

    if c < 51
      q.each do |co|
        co.set_dwc_occurrence
        r.updated.push co.id
      end
    else
      r.async = true
      q.each do |co|
        co.dwc_occurrence_update_query
      end
    end

    return r
  end

  def dwc_occurrence_update_query
    self.send(:set_dwc_occurrence)
  end

  handle_asynchronously :dwc_occurrence_update_query, run_at: Proc.new { 1.second.from_now }, queue: :query_batch_update

  # TODO: move to a helper
  def self.breakdown_status(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array

    breakdown = {
      total_objects:     collection_objects.length,
      collecting_events: {},
      determinations:    {},
      bio_overview:      []
    }

    breakdown.merge!(breakdown_buffered(collection_objects))

    collection_objects.each do |co|
      breakdown[:collecting_events].merge!(co => co.collecting_event) if co.collecting_event
      breakdown[:determinations].merge!(co => co.taxon_determinations) if co.taxon_determinations.load.any?
      breakdown[:bio_overview].push([co.total, co.biocuration_classes.collect { |a| a.name }])
    end

    breakdown
  end

  # @return [Hash]
  #   a unque list of buffered_ values observed in the collection objects passed
  def self.breakdown_buffered(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array
    breakdown = {}
    categories = BUFFERED_ATTRIBUTES

    categories.each do |c|
      breakdown[c] = []
    end

    categories.each do |c|
      collection_objects.each do |co|
        breakdown[c].push co.send(c)
      end
    end

    categories.each do |c|
      breakdown[c].uniq!
    end

    breakdown
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.earliest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:end_date_year)

    return EARLIEST_DATE if a.nil? && b.nil?  # 1700-01-01

    d = nil

    if a && b
      if a < b
        d = a
      end
    else
      d = a || b
    end
    d.to_s + '-01-01'
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.latest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:end_date_year)

    c = Time.now.strftime('%Y-%m-%d')

    return c if a.nil? && b.nil?

    d = nil

    if a && b
      if a > b
        d = a
      end
    else
      d = a || b
    end

    d.to_s + '/12/31'
  end

  # TODO: Clarify this.
  # CAREFULL - this isn't _in_, this is *with*, if it was in it would be spatial query, not a join(:geographic_items)
  #
  # Find all collection objects which have collecting events which have georeferences which have geographic_items which
  # are located within the geographic item supplied
  # @param [GeographicItem] geographic_item_id
  # @return [Scope] of CollectionObject
  def self.in_geographic_item(geographic_item, limit, steps = false)
    geographic_item_id = geographic_item.id
    if steps
      gi = GeographicItem.find(geographic_item_id)
      # find the geographic_items inside gi
      step_1 = GeographicItem.is_contained_by('any', gi) # .pluck(:id)
      # find the georeferences from the geographic_items
      step_2 = step_1.map(&:georeferences).uniq.flatten
      # find the collecting events connected to the georeferences
      step_3 = step_2.map(&:collecting_event).uniq.flatten
      # find the collection objects associated with the collecting events
      step_4 = step_3.map(&:collection_objects).flatten.map(&:id).uniq
      retval = CollectionObject.where(id: step_4.sort)
    else
      retval = CollectionObject.joins(:geographic_items)
        .where(GeographicItem.contained_by_where_sql(geographic_item.id))
        .limit(limit)
        .includes(:data_attributes, :collecting_event)
    end
    retval
  end

  # TODO: deprecate
  def self.selected_column_names
    @selected_column_names = {
      ce: {in: {}, im: {}},
      co: {in: {}, im: {}},
      bc: {in: {}, im: {}}
    } if @selected_column_names.nil?
    @selected_column_names
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collecting events
  # decode which headers to be displayed for collecting events
  def self.ce_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:ce][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:ce][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.ce_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.collecting_event.internal_attributes
      all_import_das   = collection_object.collecting_event.import_attributes
      group            = collection[:ce]
      unless group.nil?
        group.each_key { |type_key|
          group[type_key.to_sym].each_key { |header|
            this_val = nil
            case type_key.to_sym
            when :in
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            when :im
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            else
            end
          }
        }
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collection objects
  # decode which headers to be displayed for collection objects
  def self.co_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:co][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:co][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.co_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.internal_attributes
      all_import_das   = collection_object.import_attributes
      group            = collection[:co]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = nil
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
        unless group.empty?
          unless group[:im].empty?
            group[:im].each_key { |header|
              this_val = nil
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for biocuration classifications
  # decode which headers to be displayed for biocuration classifications
  def self.bc_headers(project_id)
    CollectionObject.selected_column_names
    # add selectable column names (unselected) to the column name list list
    BiocurationClass.where(project_id:).map(&:name).each { |column_name|
      @selected_column_names[:bc][:in][column_name] = {checked: '0'}
    }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.bc_attributes(collection_object, col_defs)
    retval = []
    collection = col_defs
    unless collection.nil?
      group = collection[:bc]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = collection_object.biocuration_classes.map(&:name).include?(header) ? '1' : '0'
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Array] collecting_event_ids (e.g., from CollectingEvent.in_date_range)
  # @param [Array] area_object_ids (e.g., from GeographicItem.gather_selected_data())
  # @return [Scope] of intersection of collecting events (usually by date range)
  #   and collection objects (usually by inclusion in geographic areas/items)
  def self.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id)
    collecting_events_clause = {collecting_event_id: collecting_event_ids, project: project_id}
    area_objects_clause      = {id: area_object_ids, project: project_id}

    if (collecting_event_ids.empty?)
      collecting_events_clause = {project: project_id}
    end

    if (area_object_ids.empty?)
      area_objects_clause = {}
      if (area_set)
        area_objects_clause = 'false'
      end
    end

    retval = CollectionObject.joins(:collecting_event)
      .where(collecting_events_clause)
      .where(area_objects_clause)
    retval
  end

  # TODO: move to filter
  # @param [Hash] search_start_date string in form 'yyyy-mm-dd'
  # @param [Hash] search_end_date string in form 'yyyy-mm-dd'
  # @param [Hash] partial_overlap 'on' or 'off'
  # @return [Scope] of selected collection objects through collecting events with georeferences, remember to scope to project!
  def self.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on')
    allow_partial = (partial_overlap.downcase == 'off' ? false : true) # TODO: Just get the correct values from the form!
    q = Queries::CollectingEvent::Filter.new(start_date: search_start_date, end_date: search_end_date, partial_overlap_dates: allow_partial)
    joins(:collecting_event).where(q.between_date_range_facet.to_sql)
  end

  # @param used_on [String] required, one of `TaxonDetermination`, `BiologicalAssociation`
  # @return [Scope]
  #    the max 10 most recently used collection_objects, as `used_on`
  def self.used_recently(user_id, project_id, used_on = '')
    return [] if used_on != 'TaxonDetermination' && used_on != 'BiologicalAssociation'
    t = case used_on
        when 'TaxonDetermination'
          TaxonDetermination.arel_table
        when 'BiologicalAssociation'
          BiologicalAssociation.arel_table
        end

    p = CollectionObject.arel_table

    # i is a select manager
    i = case used_on
        when 'BiologicalAssociation'
          t.project(t['biological_association_subject_id'], t['updated_at']).from(t)
            .where(
              t['updated_at'].gt(1.week.ago).and(
                t['biological_association_subject_type'].eq('CollectionObject')
              )
            )
              .where(t['updated_by_id'].eq(user_id))
              .where(t['project_id'].eq(project_id))
              .order(t['updated_at'].desc)
        else
          # TODO: update to reference new TaxonDetermination
          t.project(t['taxon_determination_object_id'], t['taxon_determination_object_type'], t['updated_at']).from(t)
            .where(t['updated_at'].gt( 1.week.ago ))
            .where(t['updated_by_id'].eq(user_id))
            .where(t['project_id'].eq(project_id))
            .order(t['updated_at'].desc)
        end

    # z is a table alias
    z = i.as('recent_t')

    j = case used_on
        when 'BiologicalAssociation'
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(
            z['biological_association_subject_id'].eq(p['id'])
          ))
        else
          # TODO: needs to be fixed to scope the taxon_determination_object_type
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(z['taxon_determination_object_id'].eq(p['id'])))
        end

    CollectionObject.joins(j).pluck(:id).uniq
  end

  # @params target [String] one of `TaxonDetermination`, `BiologicalAssociation` , nil
  # @return [Hash] otus optimized for user selection
  def self.select_optimized(user_id, project_id, target = nil)
    r = used_recently(user_id, project_id, target)
    h = {
      quick: [],
      pinboard: CollectionObject.pinned_by(user_id).where(project_id:).to_a,
      recent: []
    }

    if target && !r.empty?
      n = target.tableize.to_sym
      h[:recent] = CollectionObject.where('"collection_objects"."id" IN (?)', r.first(10) ).to_a
      h[:quick] = (CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a  +
                   CollectionObject.where('"collection_objects"."id" IN (?)', r.first(4) ).to_a).uniq
    else
      h[:recent] = CollectionObject.where(project_id:, updated_by_id: user_id).order('updated_at DESC').limit(10).to_a
      h[:quick] = CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a
    end

    h
  end

  # TODO: Unify with Extract in concern
  # @return [Identifier::Local::CatalogNumber, nil]
  #   the first (position) catalog number for this collection object, either on specimen, or container
  def preferred_catalog_number
    if i = Identifier::Local::CatalogNumber.where(identifier_object: self).order(:position).first
      i
    else
      if container
        container.identifiers.where(identifiers: {type: 'Identifier::Local::CatalogNumber'}).order(:position).first
      else
        nil
      end
    end
  end

  # @return [Identifier::Local::RecordNumber, nil]
  #   the first (position) record_Number, on a specimen
  #   !1 Doesn't presently support containers
  def preferred_record_number
    Identifier::Local::RecordNumber.where(identifier_object: self).order(:position).first
  end

  def geographic_name_classification
    # don't load the whole object, just the fields we need
    if a = DwcOccurrence.where(dwc_occurrence_object: self).select(:country, :stateProvince, :county).first

      c = a.country
      s = a.stateProvince
      y = a.county

      v = ::Utilities::Geo::DICTIONARY[c]
      c = v if v
      # s = v if v = ::Utilities::Geo::DICTIONARY[s] # None in there yet
      # y = v if v = ::Utilities::Geo::DICTIONARY[y] # None in there yet

      return {
        country: c,
        state: s,
        county: y
      }
    end
  end

  # return [Boolean]
  #    True if instance is a subclass of BiologicalCollectionObject
  def is_biological?
    self.class <= BiologicalCollectionObject ? true : false
  end

  def annotations
    h = annotations_hash
    (h['biocuration classifications'] = biocuration_classes) if is_biological? && biocuration_classifications.load.any?
    h
  end

  def sv_missing_accession_fields
    soft_validations.add(:accessioned_at, 'Date is not selected') if self.accessioned_at.nil? && !self.accession_provider.nil?
    soft_validations.add(:base, 'Provider is not selected') if !self.accessioned_at.nil? && self.accession_provider.nil?
  end

  def sv_missing_deaccession_fields
    soft_validations.add(:deaccessioned_at, 'Date is not selected') if self.deaccessioned_at.nil? && self.deaccession_reason.present?
    soft_validations.add(:base, 'Recipient is not selected') if self.deaccession_recipient.nil? && self.deaccession_reason && self.deaccessioned_at
    soft_validations.add(:deaccession_reason, 'Reason is is not defined') if self.deaccession_reason.blank? && self.deaccession_recipient && self.deaccessioned_at
  end

  def sv_missing_determination
    # see biological_collection_object
  end

  def sv_missing_collecting_event
    # see biological_collection_object
  end

  def sv_missing_preparation_type
    # see biological_collection_object
  end

  def sv_missing_repository
    # WHY? -  see biological_collection_object
  end

  def sv_missing_biocuration_classification
    # see biological_collection_object
  end

  # See Depiction#destroy_image_stub_collection_object
  # Used to determin if the CO can be
  # destroy after moving an image off
  # this object.
  def is_image_stub?
    r = [
      collecting_event_id.blank?,
      !depictions.reload.any?,
      identifiers.count <= 1,
      !taxon_determinations.any?,
      !type_materials.any?,
      !citations.any?,
      !data_attributes.any?,
      !notes.any?,
      !observations.any?
    ]

   !r.include?(false)

  end

  protected

  def collecting_event_belongs_to_project
    if collecting_event&.persisted? && (Current.project_id || project_id)
      errors.add(:base, 'collecting event is not from this project') if collecting_event.project_id != (Current.project_id || project_id)
    end
  end

  def check_that_both_of_category_and_total_are_not_present
    errors.add(:ranged_lot_category_id, 'Both ranged_lot_category and total can not be set') if ranged_lot_category_id.present? && total.present?
  end

  def check_that_either_total_or_ranged_lot_category_id_is_present
    errors.add(:base, 'Either total or a ranged lot category must be provided') if ranged_lot_category_id.blank? && total.blank?
  end

  def assign_type_if_total_or_ranged_lot_category_id_provided
    if self.total == 1
      self.type = 'Specimen'
    elsif self.total.to_i > 1
      self.type = 'Lot'
    elsif total.nil? && ranged_lot_category_id.present?
      self.type = 'RangedLot'
    end
    true
  end

  def reject_collecting_event(attributed)
    reject = true
    CollectingEvent.core_attributes.each do |a|
      if attributed[a].present?
        reject = false
        break
      end
    end
    # !! does not account for georeferences_attributes!
    reject
  end

end

#respository_id ⇒ Integer

The id of the Repository. This is an assertion of the “home” repository, i.e. where you would most reasonably find the ColletionObject when it is not “in use” by external parties. Repositories may indicate ownership, but this is inference, not an assetion. There is some notion of “custody” tied to this assertion. The assertion is only that “if this collection object was not being used, then it you can infer that it will be found in this Repository. In the absence of the assertion of a current repository it is reasonable to infer that this is also where the specimen can be currently found, however this inference will not always hold. See current_repository_id for related issues vs. modeling localization in TaxonWorks and the use of Containers.

Returns:

• (Integer)

# File 'app/models/collection_object.rb', line 64

class CollectionObject < ApplicationRecord
  include GlobalID::Identification
  include Housekeeping

  include Shared::Citations
  include Shared::Containable
  include Shared::DataAttributes
  include Shared::Loanable
  include Shared::Identifiers
  include Shared::Notes
  include Shared::Tags
  include Shared::Depictions
  include Shared::OriginRelationship
  include Shared::Confidences
  include Shared::ProtocolRelationships
  include Shared::HasPapertrail
  include Shared::Observations
  include Shared::IsData
  include Shared::QueryBatchUpdate
  include SoftValidation

  include Shared::BiologicalExtensions

  include Shared::Taxonomy # at present must be before IsDwcOccurence
  include Shared::IsDwcOccurrence
  include CollectionObject::DwcExtensions

  ignore_whitespace_on(:buffered_collecting_event, :buffered_determinations, :buffered_other_labels)

  # TODO: move to export
  CO_OTU_HEADERS = %w{OTU OTU\ name Family Genus Species Country State County Locality Latitude Longitude}.freeze

  BUFFERED_ATTRIBUTES = %i{buffered_collecting_event buffered_determinations buffered_other_labels}.freeze

  GRAPH_ENTRY_POINTS = [:biological_associations, :data_attributes, :taxon_determinations, :biocuration_classifications, :collecting_event, :origin_relationships, :extracts, :observation_matrices]

  # Identifier delegations
  # .catalog_number_cached
  delegate :cached, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true
  # .catalog_number_namespace
  delegate :namespace, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true

  # .record_number_cached
  delegate :cached, to: :preferred_record_number, prefix: :record_number, allow_nil: true
  # .record_number_namespace
  delegate :namespace, to: :preferred_record_number, prefix: :record_number, allow_nil: true

  # CollectingEvent delegations
  delegate :map_center, to: :collecting_event, prefix: :collecting_event, allow_nil: true
  delegate :collectors, to: :collecting_event, prefix: :collecting_event, allow_nil: true

  # Repository delegations
  delegate :acronym, to: :repository, prefix: :repository, allow_nil: true
  delegate :url, to: :repository, prefix: :repository, allow_nil: true
  delegate :institutional_LSID, to: :repository, prefix: :repository, allow_nil: true

  # Preparation delegations
  delegate :name, to: :preparation_type, prefix: :preparation_type, allow_nil: true

  has_one :accession_provider_role, class_name: 'AccessionProvider', as: :role_object, dependent: :destroy
  has_one :accession_provider, through: :accession_provider_role, source: :person
  has_one :deaccession_recipient_role, class_name: 'DeaccessionRecipient', as: :role_object, dependent: :destroy
  has_one :deaccession_recipient, through: :deaccession_recipient_role, source: :person

  # TODO: Deprecate these models.  Semantics also confuse with origin relationship.
  has_many :derived_collection_objects, inverse_of: :collection_object, dependent: :restrict_with_error
  has_many :collection_object_observations, through: :derived_collection_objects, inverse_of: :collection_objects

  has_many :sqed_depictions, through: :depictions, dependent: :restrict_with_error

  belongs_to :collecting_event, inverse_of: :collection_objects
  belongs_to :preparation_type, inverse_of: :collection_objects
  belongs_to :ranged_lot_category, inverse_of: :ranged_lots
  belongs_to :repository, inverse_of: :collection_objects
  belongs_to :current_repository, class_name: 'Repository', inverse_of: :collection_objects

  has_many :georeferences, through: :collecting_event
  has_many :geographic_items, through: :georeferences

  has_many :collectors, through: :collecting_event

  accepts_nested_attributes_for :collecting_event, allow_destroy: true, reject_if: :reject_collecting_event

  before_validation :assign_type_if_total_or_ranged_lot_category_id_provided

  validates_presence_of :type
  validate :check_that_either_total_or_ranged_lot_category_id_is_present
  validate :check_that_both_of_category_and_total_are_not_present
  validate :collecting_event_belongs_to_project

  soft_validate(
    :sv_missing_accession_fields,
    set: :missing_accession_fields,
    name: 'Missing accession fields',
    description: 'Name or Provider are not selected')

  soft_validate(
    :sv_missing_deaccession_fields,
    set: :missing_deaccession_fields,
    name: 'Missing deaccesson fields',
    description: 'Date, recipient, or reason are not specified')

  scope :with_sequence_name, ->(name) { joins(sequence_join_hack_sql).where(sequences: {name:}) }
  scope :via_descriptor, ->(descriptor) { joins(sequence_join_hack_sql).where(sequences: {id: descriptor.sequences}) }

  has_many :extracts, through: :origin_relationships, source: :new_object, source_type: 'Extract'
  has_many :sequences, through: :extracts

  # This is a hack, maybe related to a Rails 5.1 bug.
  # It returns the SQL that works in 5.0/4.2 that
  # links CollectionObject to Sequences:
  # joins(derived_extracts: [:derived_sequences])
  def self.sequence_join_hack_sql
    %Q{INNER JOIN  "origin_relationships"
               ON  "origin_relationships"."old_object_id" = "collection_objects"."id"
                  AND  "origin_relationships"."new_object_type" = 'Extract'
                  AND  "origin_relationships"."old_object_type" = 'CollectionObject'
       INNER JOIN  "extracts"
               ON  "extracts"."id" =  "origin_relationships"."new_object_id"
       INNER JOIN  "origin_relationships" "origin_relationships_extracts_join"
               ON  "origin_relationships_extracts_join"."old_object_id" = "extracts"."id"
                  AND  "origin_relationships_extracts_join"."new_object_type" = 'Sequence'
                  AND  "origin_relationships_extracts_join"."old_object_type" = 'Extract'
       INNER JOIN  "sequences"
               ON  "sequences"."id" = "origin_relationships_extracts_join"."new_object_id"}
  end

  def self.batch_update(params)
    request = QueryBatchRequest.new(
      async_cutoff: params[:async_cutoff] || 50,
      klass: 'CollectionObject',
      object_filter_params: params[:collection_object_query],
      object_params: params[:collection_object],
      preview: params[:preview],
    )

    request.cap = 1000

    query_batch_update(request)
  end

  def self.batch_update_dwc_occurrence(params)
    q = Queries::CollectionObject::Filter.new(params).all

    r = BatchResponse.new
    r.method = 'batch_update_dwc_occurrence'
    r.klass = 'CollectionObject'

    c = q.all.count

    if c == 0 || c > 10000
      r.cap_reason = 'Too many (or no) collection objects (max 10k)'
      return r
    end

    if c < 51
      q.each do |co|
        co.set_dwc_occurrence
        r.updated.push co.id
      end
    else
      r.async = true
      q.each do |co|
        co.dwc_occurrence_update_query
      end
    end

    return r
  end

  def dwc_occurrence_update_query
    self.send(:set_dwc_occurrence)
  end

  handle_asynchronously :dwc_occurrence_update_query, run_at: Proc.new { 1.second.from_now }, queue: :query_batch_update

  # TODO: move to a helper
  def self.breakdown_status(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array

    breakdown = {
      total_objects:     collection_objects.length,
      collecting_events: {},
      determinations:    {},
      bio_overview:      []
    }

    breakdown.merge!(breakdown_buffered(collection_objects))

    collection_objects.each do |co|
      breakdown[:collecting_events].merge!(co => co.collecting_event) if co.collecting_event
      breakdown[:determinations].merge!(co => co.taxon_determinations) if co.taxon_determinations.load.any?
      breakdown[:bio_overview].push([co.total, co.biocuration_classes.collect { |a| a.name }])
    end

    breakdown
  end

  # @return [Hash]
  #   a unque list of buffered_ values observed in the collection objects passed
  def self.breakdown_buffered(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array
    breakdown = {}
    categories = BUFFERED_ATTRIBUTES

    categories.each do |c|
      breakdown[c] = []
    end

    categories.each do |c|
      collection_objects.each do |co|
        breakdown[c].push co.send(c)
      end
    end

    categories.each do |c|
      breakdown[c].uniq!
    end

    breakdown
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.earliest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:end_date_year)

    return EARLIEST_DATE if a.nil? && b.nil?  # 1700-01-01

    d = nil

    if a && b
      if a < b
        d = a
      end
    else
      d = a || b
    end
    d.to_s + '-01-01'
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.latest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:end_date_year)

    c = Time.now.strftime('%Y-%m-%d')

    return c if a.nil? && b.nil?

    d = nil

    if a && b
      if a > b
        d = a
      end
    else
      d = a || b
    end

    d.to_s + '/12/31'
  end

  # TODO: Clarify this.
  # CAREFULL - this isn't _in_, this is *with*, if it was in it would be spatial query, not a join(:geographic_items)
  #
  # Find all collection objects which have collecting events which have georeferences which have geographic_items which
  # are located within the geographic item supplied
  # @param [GeographicItem] geographic_item_id
  # @return [Scope] of CollectionObject
  def self.in_geographic_item(geographic_item, limit, steps = false)
    geographic_item_id = geographic_item.id
    if steps
      gi = GeographicItem.find(geographic_item_id)
      # find the geographic_items inside gi
      step_1 = GeographicItem.is_contained_by('any', gi) # .pluck(:id)
      # find the georeferences from the geographic_items
      step_2 = step_1.map(&:georeferences).uniq.flatten
      # find the collecting events connected to the georeferences
      step_3 = step_2.map(&:collecting_event).uniq.flatten
      # find the collection objects associated with the collecting events
      step_4 = step_3.map(&:collection_objects).flatten.map(&:id).uniq
      retval = CollectionObject.where(id: step_4.sort)
    else
      retval = CollectionObject.joins(:geographic_items)
        .where(GeographicItem.contained_by_where_sql(geographic_item.id))
        .limit(limit)
        .includes(:data_attributes, :collecting_event)
    end
    retval
  end

  # TODO: deprecate
  def self.selected_column_names
    @selected_column_names = {
      ce: {in: {}, im: {}},
      co: {in: {}, im: {}},
      bc: {in: {}, im: {}}
    } if @selected_column_names.nil?
    @selected_column_names
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collecting events
  # decode which headers to be displayed for collecting events
  def self.ce_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:ce][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:ce][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.ce_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.collecting_event.internal_attributes
      all_import_das   = collection_object.collecting_event.import_attributes
      group            = collection[:ce]
      unless group.nil?
        group.each_key { |type_key|
          group[type_key.to_sym].each_key { |header|
            this_val = nil
            case type_key.to_sym
            when :in
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            when :im
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            else
            end
          }
        }
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collection objects
  # decode which headers to be displayed for collection objects
  def self.co_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:co][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:co][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.co_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.internal_attributes
      all_import_das   = collection_object.import_attributes
      group            = collection[:co]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = nil
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
        unless group.empty?
          unless group[:im].empty?
            group[:im].each_key { |header|
              this_val = nil
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for biocuration classifications
  # decode which headers to be displayed for biocuration classifications
  def self.bc_headers(project_id)
    CollectionObject.selected_column_names
    # add selectable column names (unselected) to the column name list list
    BiocurationClass.where(project_id:).map(&:name).each { |column_name|
      @selected_column_names[:bc][:in][column_name] = {checked: '0'}
    }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.bc_attributes(collection_object, col_defs)
    retval = []
    collection = col_defs
    unless collection.nil?
      group = collection[:bc]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = collection_object.biocuration_classes.map(&:name).include?(header) ? '1' : '0'
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Array] collecting_event_ids (e.g., from CollectingEvent.in_date_range)
  # @param [Array] area_object_ids (e.g., from GeographicItem.gather_selected_data())
  # @return [Scope] of intersection of collecting events (usually by date range)
  #   and collection objects (usually by inclusion in geographic areas/items)
  def self.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id)
    collecting_events_clause = {collecting_event_id: collecting_event_ids, project: project_id}
    area_objects_clause      = {id: area_object_ids, project: project_id}

    if (collecting_event_ids.empty?)
      collecting_events_clause = {project: project_id}
    end

    if (area_object_ids.empty?)
      area_objects_clause = {}
      if (area_set)
        area_objects_clause = 'false'
      end
    end

    retval = CollectionObject.joins(:collecting_event)
      .where(collecting_events_clause)
      .where(area_objects_clause)
    retval
  end

  # TODO: move to filter
  # @param [Hash] search_start_date string in form 'yyyy-mm-dd'
  # @param [Hash] search_end_date string in form 'yyyy-mm-dd'
  # @param [Hash] partial_overlap 'on' or 'off'
  # @return [Scope] of selected collection objects through collecting events with georeferences, remember to scope to project!
  def self.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on')
    allow_partial = (partial_overlap.downcase == 'off' ? false : true) # TODO: Just get the correct values from the form!
    q = Queries::CollectingEvent::Filter.new(start_date: search_start_date, end_date: search_end_date, partial_overlap_dates: allow_partial)
    joins(:collecting_event).where(q.between_date_range_facet.to_sql)
  end

  # @param used_on [String] required, one of `TaxonDetermination`, `BiologicalAssociation`
  # @return [Scope]
  #    the max 10 most recently used collection_objects, as `used_on`
  def self.used_recently(user_id, project_id, used_on = '')
    return [] if used_on != 'TaxonDetermination' && used_on != 'BiologicalAssociation'
    t = case used_on
        when 'TaxonDetermination'
          TaxonDetermination.arel_table
        when 'BiologicalAssociation'
          BiologicalAssociation.arel_table
        end

    p = CollectionObject.arel_table

    # i is a select manager
    i = case used_on
        when 'BiologicalAssociation'
          t.project(t['biological_association_subject_id'], t['updated_at']).from(t)
            .where(
              t['updated_at'].gt(1.week.ago).and(
                t['biological_association_subject_type'].eq('CollectionObject')
              )
            )
              .where(t['updated_by_id'].eq(user_id))
              .where(t['project_id'].eq(project_id))
              .order(t['updated_at'].desc)
        else
          # TODO: update to reference new TaxonDetermination
          t.project(t['taxon_determination_object_id'], t['taxon_determination_object_type'], t['updated_at']).from(t)
            .where(t['updated_at'].gt( 1.week.ago ))
            .where(t['updated_by_id'].eq(user_id))
            .where(t['project_id'].eq(project_id))
            .order(t['updated_at'].desc)
        end

    # z is a table alias
    z = i.as('recent_t')

    j = case used_on
        when 'BiologicalAssociation'
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(
            z['biological_association_subject_id'].eq(p['id'])
          ))
        else
          # TODO: needs to be fixed to scope the taxon_determination_object_type
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(z['taxon_determination_object_id'].eq(p['id'])))
        end

    CollectionObject.joins(j).pluck(:id).uniq
  end

  # @params target [String] one of `TaxonDetermination`, `BiologicalAssociation` , nil
  # @return [Hash] otus optimized for user selection
  def self.select_optimized(user_id, project_id, target = nil)
    r = used_recently(user_id, project_id, target)
    h = {
      quick: [],
      pinboard: CollectionObject.pinned_by(user_id).where(project_id:).to_a,
      recent: []
    }

    if target && !r.empty?
      n = target.tableize.to_sym
      h[:recent] = CollectionObject.where('"collection_objects"."id" IN (?)', r.first(10) ).to_a
      h[:quick] = (CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a  +
                   CollectionObject.where('"collection_objects"."id" IN (?)', r.first(4) ).to_a).uniq
    else
      h[:recent] = CollectionObject.where(project_id:, updated_by_id: user_id).order('updated_at DESC').limit(10).to_a
      h[:quick] = CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a
    end

    h
  end

  # TODO: Unify with Extract in concern
  # @return [Identifier::Local::CatalogNumber, nil]
  #   the first (position) catalog number for this collection object, either on specimen, or container
  def preferred_catalog_number
    if i = Identifier::Local::CatalogNumber.where(identifier_object: self).order(:position).first
      i
    else
      if container
        container.identifiers.where(identifiers: {type: 'Identifier::Local::CatalogNumber'}).order(:position).first
      else
        nil
      end
    end
  end

  # @return [Identifier::Local::RecordNumber, nil]
  #   the first (position) record_Number, on a specimen
  #   !1 Doesn't presently support containers
  def preferred_record_number
    Identifier::Local::RecordNumber.where(identifier_object: self).order(:position).first
  end

  def geographic_name_classification
    # don't load the whole object, just the fields we need
    if a = DwcOccurrence.where(dwc_occurrence_object: self).select(:country, :stateProvince, :county).first

      c = a.country
      s = a.stateProvince
      y = a.county

      v = ::Utilities::Geo::DICTIONARY[c]
      c = v if v
      # s = v if v = ::Utilities::Geo::DICTIONARY[s] # None in there yet
      # y = v if v = ::Utilities::Geo::DICTIONARY[y] # None in there yet

      return {
        country: c,
        state: s,
        county: y
      }
    end
  end

  # return [Boolean]
  #    True if instance is a subclass of BiologicalCollectionObject
  def is_biological?
    self.class <= BiologicalCollectionObject ? true : false
  end

  def annotations
    h = annotations_hash
    (h['biocuration classifications'] = biocuration_classes) if is_biological? && biocuration_classifications.load.any?
    h
  end

  def sv_missing_accession_fields
    soft_validations.add(:accessioned_at, 'Date is not selected') if self.accessioned_at.nil? && !self.accession_provider.nil?
    soft_validations.add(:base, 'Provider is not selected') if !self.accessioned_at.nil? && self.accession_provider.nil?
  end

  def sv_missing_deaccession_fields
    soft_validations.add(:deaccessioned_at, 'Date is not selected') if self.deaccessioned_at.nil? && self.deaccession_reason.present?
    soft_validations.add(:base, 'Recipient is not selected') if self.deaccession_recipient.nil? && self.deaccession_reason && self.deaccessioned_at
    soft_validations.add(:deaccession_reason, 'Reason is is not defined') if self.deaccession_reason.blank? && self.deaccession_recipient && self.deaccessioned_at
  end

  def sv_missing_determination
    # see biological_collection_object
  end

  def sv_missing_collecting_event
    # see biological_collection_object
  end

  def sv_missing_preparation_type
    # see biological_collection_object
  end

  def sv_missing_repository
    # WHY? -  see biological_collection_object
  end

  def sv_missing_biocuration_classification
    # see biological_collection_object
  end

  # See Depiction#destroy_image_stub_collection_object
  # Used to determin if the CO can be
  # destroy after moving an image off
  # this object.
  def is_image_stub?
    r = [
      collecting_event_id.blank?,
      !depictions.reload.any?,
      identifiers.count <= 1,
      !taxon_determinations.any?,
      !type_materials.any?,
      !citations.any?,
      !data_attributes.any?,
      !notes.any?,
      !observations.any?
    ]

   !r.include?(false)

  end

  protected

  def collecting_event_belongs_to_project
    if collecting_event&.persisted? && (Current.project_id || project_id)
      errors.add(:base, 'collecting event is not from this project') if collecting_event.project_id != (Current.project_id || project_id)
    end
  end

  def check_that_both_of_category_and_total_are_not_present
    errors.add(:ranged_lot_category_id, 'Both ranged_lot_category and total can not be set') if ranged_lot_category_id.present? && total.present?
  end

  def check_that_either_total_or_ranged_lot_category_id_is_present
    errors.add(:base, 'Either total or a ranged lot category must be provided') if ranged_lot_category_id.blank? && total.blank?
  end

  def assign_type_if_total_or_ranged_lot_category_id_provided
    if self.total == 1
      self.type = 'Specimen'
    elsif self.total.to_i > 1
      self.type = 'Lot'
    elsif total.nil? && ranged_lot_category_id.present?
      self.type = 'RangedLot'
    end
    true
  end

  def reject_collecting_event(attributed)
    reject = true
    CollectingEvent.core_attributes.each do |a|
      if attributed[a].present?
        reject = false
        break
      end
    end
    # !! does not account for georeferences_attributes!
    reject
  end

end

#total ⇒ Integer

The enumerated number of things, as asserted by the person managing the record. Different totals will default to different subclasses. How you enumerate your collection objects is up to you. If you want to call one chunk of coral 50 things, that’s fine (total = 50), if you want to call one coral one thing (total = 1) that’s fine too. If not nil then ranged_lot_category_id must be nil. When =1 the subclass is Specimen, when > 1 the subclass is Lot.

Returns:

• (Integer)

# File 'app/models/collection_object.rb', line 64

class CollectionObject < ApplicationRecord
  include GlobalID::Identification
  include Housekeeping

  include Shared::Citations
  include Shared::Containable
  include Shared::DataAttributes
  include Shared::Loanable
  include Shared::Identifiers
  include Shared::Notes
  include Shared::Tags
  include Shared::Depictions
  include Shared::OriginRelationship
  include Shared::Confidences
  include Shared::ProtocolRelationships
  include Shared::HasPapertrail
  include Shared::Observations
  include Shared::IsData
  include Shared::QueryBatchUpdate
  include SoftValidation

  include Shared::BiologicalExtensions

  include Shared::Taxonomy # at present must be before IsDwcOccurence
  include Shared::IsDwcOccurrence
  include CollectionObject::DwcExtensions

  ignore_whitespace_on(:buffered_collecting_event, :buffered_determinations, :buffered_other_labels)

  # TODO: move to export
  CO_OTU_HEADERS = %w{OTU OTU\ name Family Genus Species Country State County Locality Latitude Longitude}.freeze

  BUFFERED_ATTRIBUTES = %i{buffered_collecting_event buffered_determinations buffered_other_labels}.freeze

  GRAPH_ENTRY_POINTS = [:biological_associations, :data_attributes, :taxon_determinations, :biocuration_classifications, :collecting_event, :origin_relationships, :extracts, :observation_matrices]

  # Identifier delegations
  # .catalog_number_cached
  delegate :cached, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true
  # .catalog_number_namespace
  delegate :namespace, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true

  # .record_number_cached
  delegate :cached, to: :preferred_record_number, prefix: :record_number, allow_nil: true
  # .record_number_namespace
  delegate :namespace, to: :preferred_record_number, prefix: :record_number, allow_nil: true

  # CollectingEvent delegations
  delegate :map_center, to: :collecting_event, prefix: :collecting_event, allow_nil: true
  delegate :collectors, to: :collecting_event, prefix: :collecting_event, allow_nil: true

  # Repository delegations
  delegate :acronym, to: :repository, prefix: :repository, allow_nil: true
  delegate :url, to: :repository, prefix: :repository, allow_nil: true
  delegate :institutional_LSID, to: :repository, prefix: :repository, allow_nil: true

  # Preparation delegations
  delegate :name, to: :preparation_type, prefix: :preparation_type, allow_nil: true

  has_one :accession_provider_role, class_name: 'AccessionProvider', as: :role_object, dependent: :destroy
  has_one :accession_provider, through: :accession_provider_role, source: :person
  has_one :deaccession_recipient_role, class_name: 'DeaccessionRecipient', as: :role_object, dependent: :destroy
  has_one :deaccession_recipient, through: :deaccession_recipient_role, source: :person

  # TODO: Deprecate these models.  Semantics also confuse with origin relationship.
  has_many :derived_collection_objects, inverse_of: :collection_object, dependent: :restrict_with_error
  has_many :collection_object_observations, through: :derived_collection_objects, inverse_of: :collection_objects

  has_many :sqed_depictions, through: :depictions, dependent: :restrict_with_error

  belongs_to :collecting_event, inverse_of: :collection_objects
  belongs_to :preparation_type, inverse_of: :collection_objects
  belongs_to :ranged_lot_category, inverse_of: :ranged_lots
  belongs_to :repository, inverse_of: :collection_objects
  belongs_to :current_repository, class_name: 'Repository', inverse_of: :collection_objects

  has_many :georeferences, through: :collecting_event
  has_many :geographic_items, through: :georeferences

  has_many :collectors, through: :collecting_event

  accepts_nested_attributes_for :collecting_event, allow_destroy: true, reject_if: :reject_collecting_event

  before_validation :assign_type_if_total_or_ranged_lot_category_id_provided

  validates_presence_of :type
  validate :check_that_either_total_or_ranged_lot_category_id_is_present
  validate :check_that_both_of_category_and_total_are_not_present
  validate :collecting_event_belongs_to_project

  soft_validate(
    :sv_missing_accession_fields,
    set: :missing_accession_fields,
    name: 'Missing accession fields',
    description: 'Name or Provider are not selected')

  soft_validate(
    :sv_missing_deaccession_fields,
    set: :missing_deaccession_fields,
    name: 'Missing deaccesson fields',
    description: 'Date, recipient, or reason are not specified')

  scope :with_sequence_name, ->(name) { joins(sequence_join_hack_sql).where(sequences: {name:}) }
  scope :via_descriptor, ->(descriptor) { joins(sequence_join_hack_sql).where(sequences: {id: descriptor.sequences}) }

  has_many :extracts, through: :origin_relationships, source: :new_object, source_type: 'Extract'
  has_many :sequences, through: :extracts

  # This is a hack, maybe related to a Rails 5.1 bug.
  # It returns the SQL that works in 5.0/4.2 that
  # links CollectionObject to Sequences:
  # joins(derived_extracts: [:derived_sequences])
  def self.sequence_join_hack_sql
    %Q{INNER JOIN  "origin_relationships"
               ON  "origin_relationships"."old_object_id" = "collection_objects"."id"
                  AND  "origin_relationships"."new_object_type" = 'Extract'
                  AND  "origin_relationships"."old_object_type" = 'CollectionObject'
       INNER JOIN  "extracts"
               ON  "extracts"."id" =  "origin_relationships"."new_object_id"
       INNER JOIN  "origin_relationships" "origin_relationships_extracts_join"
               ON  "origin_relationships_extracts_join"."old_object_id" = "extracts"."id"
                  AND  "origin_relationships_extracts_join"."new_object_type" = 'Sequence'
                  AND  "origin_relationships_extracts_join"."old_object_type" = 'Extract'
       INNER JOIN  "sequences"
               ON  "sequences"."id" = "origin_relationships_extracts_join"."new_object_id"}
  end

  def self.batch_update(params)
    request = QueryBatchRequest.new(
      async_cutoff: params[:async_cutoff] || 50,
      klass: 'CollectionObject',
      object_filter_params: params[:collection_object_query],
      object_params: params[:collection_object],
      preview: params[:preview],
    )

    request.cap = 1000

    query_batch_update(request)
  end

  def self.batch_update_dwc_occurrence(params)
    q = Queries::CollectionObject::Filter.new(params).all

    r = BatchResponse.new
    r.method = 'batch_update_dwc_occurrence'
    r.klass = 'CollectionObject'

    c = q.all.count

    if c == 0 || c > 10000
      r.cap_reason = 'Too many (or no) collection objects (max 10k)'
      return r
    end

    if c < 51
      q.each do |co|
        co.set_dwc_occurrence
        r.updated.push co.id
      end
    else
      r.async = true
      q.each do |co|
        co.dwc_occurrence_update_query
      end
    end

    return r
  end

  def dwc_occurrence_update_query
    self.send(:set_dwc_occurrence)
  end

  handle_asynchronously :dwc_occurrence_update_query, run_at: Proc.new { 1.second.from_now }, queue: :query_batch_update

  # TODO: move to a helper
  def self.breakdown_status(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array

    breakdown = {
      total_objects:     collection_objects.length,
      collecting_events: {},
      determinations:    {},
      bio_overview:      []
    }

    breakdown.merge!(breakdown_buffered(collection_objects))

    collection_objects.each do |co|
      breakdown[:collecting_events].merge!(co => co.collecting_event) if co.collecting_event
      breakdown[:determinations].merge!(co => co.taxon_determinations) if co.taxon_determinations.load.any?
      breakdown[:bio_overview].push([co.total, co.biocuration_classes.collect { |a| a.name }])
    end

    breakdown
  end

  # @return [Hash]
  #   a unque list of buffered_ values observed in the collection objects passed
  def self.breakdown_buffered(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array
    breakdown = {}
    categories = BUFFERED_ATTRIBUTES

    categories.each do |c|
      breakdown[c] = []
    end

    categories.each do |c|
      collection_objects.each do |co|
        breakdown[c].push co.send(c)
      end
    end

    categories.each do |c|
      breakdown[c].uniq!
    end

    breakdown
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.earliest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:end_date_year)

    return EARLIEST_DATE if a.nil? && b.nil?  # 1700-01-01

    d = nil

    if a && b
      if a < b
        d = a
      end
    else
      d = a || b
    end
    d.to_s + '-01-01'
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.latest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:end_date_year)

    c = Time.now.strftime('%Y-%m-%d')

    return c if a.nil? && b.nil?

    d = nil

    if a && b
      if a > b
        d = a
      end
    else
      d = a || b
    end

    d.to_s + '/12/31'
  end

  # TODO: Clarify this.
  # CAREFULL - this isn't _in_, this is *with*, if it was in it would be spatial query, not a join(:geographic_items)
  #
  # Find all collection objects which have collecting events which have georeferences which have geographic_items which
  # are located within the geographic item supplied
  # @param [GeographicItem] geographic_item_id
  # @return [Scope] of CollectionObject
  def self.in_geographic_item(geographic_item, limit, steps = false)
    geographic_item_id = geographic_item.id
    if steps
      gi = GeographicItem.find(geographic_item_id)
      # find the geographic_items inside gi
      step_1 = GeographicItem.is_contained_by('any', gi) # .pluck(:id)
      # find the georeferences from the geographic_items
      step_2 = step_1.map(&:georeferences).uniq.flatten
      # find the collecting events connected to the georeferences
      step_3 = step_2.map(&:collecting_event).uniq.flatten
      # find the collection objects associated with the collecting events
      step_4 = step_3.map(&:collection_objects).flatten.map(&:id).uniq
      retval = CollectionObject.where(id: step_4.sort)
    else
      retval = CollectionObject.joins(:geographic_items)
        .where(GeographicItem.contained_by_where_sql(geographic_item.id))
        .limit(limit)
        .includes(:data_attributes, :collecting_event)
    end
    retval
  end

  # TODO: deprecate
  def self.selected_column_names
    @selected_column_names = {
      ce: {in: {}, im: {}},
      co: {in: {}, im: {}},
      bc: {in: {}, im: {}}
    } if @selected_column_names.nil?
    @selected_column_names
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collecting events
  # decode which headers to be displayed for collecting events
  def self.ce_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:ce][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:ce][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.ce_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.collecting_event.internal_attributes
      all_import_das   = collection_object.collecting_event.import_attributes
      group            = collection[:ce]
      unless group.nil?
        group.each_key { |type_key|
          group[type_key.to_sym].each_key { |header|
            this_val = nil
            case type_key.to_sym
            when :in
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            when :im
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            else
            end
          }
        }
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collection objects
  # decode which headers to be displayed for collection objects
  def self.co_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:co][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:co][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.co_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.internal_attributes
      all_import_das   = collection_object.import_attributes
      group            = collection[:co]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = nil
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
        unless group.empty?
          unless group[:im].empty?
            group[:im].each_key { |header|
              this_val = nil
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for biocuration classifications
  # decode which headers to be displayed for biocuration classifications
  def self.bc_headers(project_id)
    CollectionObject.selected_column_names
    # add selectable column names (unselected) to the column name list list
    BiocurationClass.where(project_id:).map(&:name).each { |column_name|
      @selected_column_names[:bc][:in][column_name] = {checked: '0'}
    }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.bc_attributes(collection_object, col_defs)
    retval = []
    collection = col_defs
    unless collection.nil?
      group = collection[:bc]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = collection_object.biocuration_classes.map(&:name).include?(header) ? '1' : '0'
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Array] collecting_event_ids (e.g., from CollectingEvent.in_date_range)
  # @param [Array] area_object_ids (e.g., from GeographicItem.gather_selected_data())
  # @return [Scope] of intersection of collecting events (usually by date range)
  #   and collection objects (usually by inclusion in geographic areas/items)
  def self.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id)
    collecting_events_clause = {collecting_event_id: collecting_event_ids, project: project_id}
    area_objects_clause      = {id: area_object_ids, project: project_id}

    if (collecting_event_ids.empty?)
      collecting_events_clause = {project: project_id}
    end

    if (area_object_ids.empty?)
      area_objects_clause = {}
      if (area_set)
        area_objects_clause = 'false'
      end
    end

    retval = CollectionObject.joins(:collecting_event)
      .where(collecting_events_clause)
      .where(area_objects_clause)
    retval
  end

  # TODO: move to filter
  # @param [Hash] search_start_date string in form 'yyyy-mm-dd'
  # @param [Hash] search_end_date string in form 'yyyy-mm-dd'
  # @param [Hash] partial_overlap 'on' or 'off'
  # @return [Scope] of selected collection objects through collecting events with georeferences, remember to scope to project!
  def self.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on')
    allow_partial = (partial_overlap.downcase == 'off' ? false : true) # TODO: Just get the correct values from the form!
    q = Queries::CollectingEvent::Filter.new(start_date: search_start_date, end_date: search_end_date, partial_overlap_dates: allow_partial)
    joins(:collecting_event).where(q.between_date_range_facet.to_sql)
  end

  # @param used_on [String] required, one of `TaxonDetermination`, `BiologicalAssociation`
  # @return [Scope]
  #    the max 10 most recently used collection_objects, as `used_on`
  def self.used_recently(user_id, project_id, used_on = '')
    return [] if used_on != 'TaxonDetermination' && used_on != 'BiologicalAssociation'
    t = case used_on
        when 'TaxonDetermination'
          TaxonDetermination.arel_table
        when 'BiologicalAssociation'
          BiologicalAssociation.arel_table
        end

    p = CollectionObject.arel_table

    # i is a select manager
    i = case used_on
        when 'BiologicalAssociation'
          t.project(t['biological_association_subject_id'], t['updated_at']).from(t)
            .where(
              t['updated_at'].gt(1.week.ago).and(
                t['biological_association_subject_type'].eq('CollectionObject')
              )
            )
              .where(t['updated_by_id'].eq(user_id))
              .where(t['project_id'].eq(project_id))
              .order(t['updated_at'].desc)
        else
          # TODO: update to reference new TaxonDetermination
          t.project(t['taxon_determination_object_id'], t['taxon_determination_object_type'], t['updated_at']).from(t)
            .where(t['updated_at'].gt( 1.week.ago ))
            .where(t['updated_by_id'].eq(user_id))
            .where(t['project_id'].eq(project_id))
            .order(t['updated_at'].desc)
        end

    # z is a table alias
    z = i.as('recent_t')

    j = case used_on
        when 'BiologicalAssociation'
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(
            z['biological_association_subject_id'].eq(p['id'])
          ))
        else
          # TODO: needs to be fixed to scope the taxon_determination_object_type
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(z['taxon_determination_object_id'].eq(p['id'])))
        end

    CollectionObject.joins(j).pluck(:id).uniq
  end

  # @params target [String] one of `TaxonDetermination`, `BiologicalAssociation` , nil
  # @return [Hash] otus optimized for user selection
  def self.select_optimized(user_id, project_id, target = nil)
    r = used_recently(user_id, project_id, target)
    h = {
      quick: [],
      pinboard: CollectionObject.pinned_by(user_id).where(project_id:).to_a,
      recent: []
    }

    if target && !r.empty?
      n = target.tableize.to_sym
      h[:recent] = CollectionObject.where('"collection_objects"."id" IN (?)', r.first(10) ).to_a
      h[:quick] = (CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a  +
                   CollectionObject.where('"collection_objects"."id" IN (?)', r.first(4) ).to_a).uniq
    else
      h[:recent] = CollectionObject.where(project_id:, updated_by_id: user_id).order('updated_at DESC').limit(10).to_a
      h[:quick] = CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a
    end

    h
  end

  # TODO: Unify with Extract in concern
  # @return [Identifier::Local::CatalogNumber, nil]
  #   the first (position) catalog number for this collection object, either on specimen, or container
  def preferred_catalog_number
    if i = Identifier::Local::CatalogNumber.where(identifier_object: self).order(:position).first
      i
    else
      if container
        container.identifiers.where(identifiers: {type: 'Identifier::Local::CatalogNumber'}).order(:position).first
      else
        nil
      end
    end
  end

  # @return [Identifier::Local::RecordNumber, nil]
  #   the first (position) record_Number, on a specimen
  #   !1 Doesn't presently support containers
  def preferred_record_number
    Identifier::Local::RecordNumber.where(identifier_object: self).order(:position).first
  end

  def geographic_name_classification
    # don't load the whole object, just the fields we need
    if a = DwcOccurrence.where(dwc_occurrence_object: self).select(:country, :stateProvince, :county).first

      c = a.country
      s = a.stateProvince
      y = a.county

      v = ::Utilities::Geo::DICTIONARY[c]
      c = v if v
      # s = v if v = ::Utilities::Geo::DICTIONARY[s] # None in there yet
      # y = v if v = ::Utilities::Geo::DICTIONARY[y] # None in there yet

      return {
        country: c,
        state: s,
        county: y
      }
    end
  end

  # return [Boolean]
  #    True if instance is a subclass of BiologicalCollectionObject
  def is_biological?
    self.class <= BiologicalCollectionObject ? true : false
  end

  def annotations
    h = annotations_hash
    (h['biocuration classifications'] = biocuration_classes) if is_biological? && biocuration_classifications.load.any?
    h
  end

  def sv_missing_accession_fields
    soft_validations.add(:accessioned_at, 'Date is not selected') if self.accessioned_at.nil? && !self.accession_provider.nil?
    soft_validations.add(:base, 'Provider is not selected') if !self.accessioned_at.nil? && self.accession_provider.nil?
  end

  def sv_missing_deaccession_fields
    soft_validations.add(:deaccessioned_at, 'Date is not selected') if self.deaccessioned_at.nil? && self.deaccession_reason.present?
    soft_validations.add(:base, 'Recipient is not selected') if self.deaccession_recipient.nil? && self.deaccession_reason && self.deaccessioned_at
    soft_validations.add(:deaccession_reason, 'Reason is is not defined') if self.deaccession_reason.blank? && self.deaccession_recipient && self.deaccessioned_at
  end

  def sv_missing_determination
    # see biological_collection_object
  end

  def sv_missing_collecting_event
    # see biological_collection_object
  end

  def sv_missing_preparation_type
    # see biological_collection_object
  end

  def sv_missing_repository
    # WHY? -  see biological_collection_object
  end

  def sv_missing_biocuration_classification
    # see biological_collection_object
  end

  # See Depiction#destroy_image_stub_collection_object
  # Used to determin if the CO can be
  # destroy after moving an image off
  # this object.
  def is_image_stub?
    r = [
      collecting_event_id.blank?,
      !depictions.reload.any?,
      identifiers.count <= 1,
      !taxon_determinations.any?,
      !type_materials.any?,
      !citations.any?,
      !data_attributes.any?,
      !notes.any?,
      !observations.any?
    ]

   !r.include?(false)

  end

  protected

  def collecting_event_belongs_to_project
    if collecting_event&.persisted? && (Current.project_id || project_id)
      errors.add(:base, 'collecting event is not from this project') if collecting_event.project_id != (Current.project_id || project_id)
    end
  end

  def check_that_both_of_category_and_total_are_not_present
    errors.add(:ranged_lot_category_id, 'Both ranged_lot_category and total can not be set') if ranged_lot_category_id.present? && total.present?
  end

  def check_that_either_total_or_ranged_lot_category_id_is_present
    errors.add(:base, 'Either total or a ranged lot category must be provided') if ranged_lot_category_id.blank? && total.blank?
  end

  def assign_type_if_total_or_ranged_lot_category_id_provided
    if self.total == 1
      self.type = 'Specimen'
    elsif self.total.to_i > 1
      self.type = 'Lot'
    elsif total.nil? && ranged_lot_category_id.present?
      self.type = 'RangedLot'
    end
    true
  end

  def reject_collecting_event(attributed)
    reject = true
    CollectingEvent.core_attributes.each do |a|
      if attributed[a].present?
        reject = false
        break
      end
    end
    # !! does not account for georeferences_attributes!
    reject
  end

end

#type ⇒ String

Returns the subclass of collection object, e.g. Specimen, Lot, or RangedLot.

Returns:

• (String) —

  the subclass of collection object, e.g. Specimen, Lot, or RangedLot

# File 'app/models/collection_object.rb', line 64

class CollectionObject < ApplicationRecord
  include GlobalID::Identification
  include Housekeeping

  include Shared::Citations
  include Shared::Containable
  include Shared::DataAttributes
  include Shared::Loanable
  include Shared::Identifiers
  include Shared::Notes
  include Shared::Tags
  include Shared::Depictions
  include Shared::OriginRelationship
  include Shared::Confidences
  include Shared::ProtocolRelationships
  include Shared::HasPapertrail
  include Shared::Observations
  include Shared::IsData
  include Shared::QueryBatchUpdate
  include SoftValidation

  include Shared::BiologicalExtensions

  include Shared::Taxonomy # at present must be before IsDwcOccurence
  include Shared::IsDwcOccurrence
  include CollectionObject::DwcExtensions

  ignore_whitespace_on(:buffered_collecting_event, :buffered_determinations, :buffered_other_labels)

  # TODO: move to export
  CO_OTU_HEADERS = %w{OTU OTU\ name Family Genus Species Country State County Locality Latitude Longitude}.freeze

  BUFFERED_ATTRIBUTES = %i{buffered_collecting_event buffered_determinations buffered_other_labels}.freeze

  GRAPH_ENTRY_POINTS = [:biological_associations, :data_attributes, :taxon_determinations, :biocuration_classifications, :collecting_event, :origin_relationships, :extracts, :observation_matrices]

  # Identifier delegations
  # .catalog_number_cached
  delegate :cached, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true
  # .catalog_number_namespace
  delegate :namespace, to: :preferred_catalog_number, prefix: :catalog_number, allow_nil: true

  # .record_number_cached
  delegate :cached, to: :preferred_record_number, prefix: :record_number, allow_nil: true
  # .record_number_namespace
  delegate :namespace, to: :preferred_record_number, prefix: :record_number, allow_nil: true

  # CollectingEvent delegations
  delegate :map_center, to: :collecting_event, prefix: :collecting_event, allow_nil: true
  delegate :collectors, to: :collecting_event, prefix: :collecting_event, allow_nil: true

  # Repository delegations
  delegate :acronym, to: :repository, prefix: :repository, allow_nil: true
  delegate :url, to: :repository, prefix: :repository, allow_nil: true
  delegate :institutional_LSID, to: :repository, prefix: :repository, allow_nil: true

  # Preparation delegations
  delegate :name, to: :preparation_type, prefix: :preparation_type, allow_nil: true

  has_one :accession_provider_role, class_name: 'AccessionProvider', as: :role_object, dependent: :destroy
  has_one :accession_provider, through: :accession_provider_role, source: :person
  has_one :deaccession_recipient_role, class_name: 'DeaccessionRecipient', as: :role_object, dependent: :destroy
  has_one :deaccession_recipient, through: :deaccession_recipient_role, source: :person

  # TODO: Deprecate these models.  Semantics also confuse with origin relationship.
  has_many :derived_collection_objects, inverse_of: :collection_object, dependent: :restrict_with_error
  has_many :collection_object_observations, through: :derived_collection_objects, inverse_of: :collection_objects

  has_many :sqed_depictions, through: :depictions, dependent: :restrict_with_error

  belongs_to :collecting_event, inverse_of: :collection_objects
  belongs_to :preparation_type, inverse_of: :collection_objects
  belongs_to :ranged_lot_category, inverse_of: :ranged_lots
  belongs_to :repository, inverse_of: :collection_objects
  belongs_to :current_repository, class_name: 'Repository', inverse_of: :collection_objects

  has_many :georeferences, through: :collecting_event
  has_many :geographic_items, through: :georeferences

  has_many :collectors, through: :collecting_event

  accepts_nested_attributes_for :collecting_event, allow_destroy: true, reject_if: :reject_collecting_event

  before_validation :assign_type_if_total_or_ranged_lot_category_id_provided

  validates_presence_of :type
  validate :check_that_either_total_or_ranged_lot_category_id_is_present
  validate :check_that_both_of_category_and_total_are_not_present
  validate :collecting_event_belongs_to_project

  soft_validate(
    :sv_missing_accession_fields,
    set: :missing_accession_fields,
    name: 'Missing accession fields',
    description: 'Name or Provider are not selected')

  soft_validate(
    :sv_missing_deaccession_fields,
    set: :missing_deaccession_fields,
    name: 'Missing deaccesson fields',
    description: 'Date, recipient, or reason are not specified')

  scope :with_sequence_name, ->(name) { joins(sequence_join_hack_sql).where(sequences: {name:}) }
  scope :via_descriptor, ->(descriptor) { joins(sequence_join_hack_sql).where(sequences: {id: descriptor.sequences}) }

  has_many :extracts, through: :origin_relationships, source: :new_object, source_type: 'Extract'
  has_many :sequences, through: :extracts

  # This is a hack, maybe related to a Rails 5.1 bug.
  # It returns the SQL that works in 5.0/4.2 that
  # links CollectionObject to Sequences:
  # joins(derived_extracts: [:derived_sequences])
  def self.sequence_join_hack_sql
    %Q{INNER JOIN  "origin_relationships"
               ON  "origin_relationships"."old_object_id" = "collection_objects"."id"
                  AND  "origin_relationships"."new_object_type" = 'Extract'
                  AND  "origin_relationships"."old_object_type" = 'CollectionObject'
       INNER JOIN  "extracts"
               ON  "extracts"."id" =  "origin_relationships"."new_object_id"
       INNER JOIN  "origin_relationships" "origin_relationships_extracts_join"
               ON  "origin_relationships_extracts_join"."old_object_id" = "extracts"."id"
                  AND  "origin_relationships_extracts_join"."new_object_type" = 'Sequence'
                  AND  "origin_relationships_extracts_join"."old_object_type" = 'Extract'
       INNER JOIN  "sequences"
               ON  "sequences"."id" = "origin_relationships_extracts_join"."new_object_id"}
  end

  def self.batch_update(params)
    request = QueryBatchRequest.new(
      async_cutoff: params[:async_cutoff] || 50,
      klass: 'CollectionObject',
      object_filter_params: params[:collection_object_query],
      object_params: params[:collection_object],
      preview: params[:preview],
    )

    request.cap = 1000

    query_batch_update(request)
  end

  def self.batch_update_dwc_occurrence(params)
    q = Queries::CollectionObject::Filter.new(params).all

    r = BatchResponse.new
    r.method = 'batch_update_dwc_occurrence'
    r.klass = 'CollectionObject'

    c = q.all.count

    if c == 0 || c > 10000
      r.cap_reason = 'Too many (or no) collection objects (max 10k)'
      return r
    end

    if c < 51
      q.each do |co|
        co.set_dwc_occurrence
        r.updated.push co.id
      end
    else
      r.async = true
      q.each do |co|
        co.dwc_occurrence_update_query
      end
    end

    return r
  end

  def dwc_occurrence_update_query
    self.send(:set_dwc_occurrence)
  end

  handle_asynchronously :dwc_occurrence_update_query, run_at: Proc.new { 1.second.from_now }, queue: :query_batch_update

  # TODO: move to a helper
  def self.breakdown_status(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array

    breakdown = {
      total_objects:     collection_objects.length,
      collecting_events: {},
      determinations:    {},
      bio_overview:      []
    }

    breakdown.merge!(breakdown_buffered(collection_objects))

    collection_objects.each do |co|
      breakdown[:collecting_events].merge!(co => co.collecting_event) if co.collecting_event
      breakdown[:determinations].merge!(co => co.taxon_determinations) if co.taxon_determinations.load.any?
      breakdown[:bio_overview].push([co.total, co.biocuration_classes.collect { |a| a.name }])
    end

    breakdown
  end

  # @return [Hash]
  #   a unque list of buffered_ values observed in the collection objects passed
  def self.breakdown_buffered(collection_objects)
    collection_objects = [collection_objects] if collection_objects.class != Array
    breakdown = {}
    categories = BUFFERED_ATTRIBUTES

    categories.each do |c|
      breakdown[c] = []
    end

    categories.each do |c|
      collection_objects.each do |co|
        breakdown[c].push co.send(c)
      end
    end

    categories.each do |c|
      breakdown[c].uniq!
    end

    breakdown
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.earliest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:end_date_year)

    return EARLIEST_DATE if a.nil? && b.nil?  # 1700-01-01

    d = nil

    if a && b
      if a < b
        d = a
      end
    else
      d = a || b
    end
    d.to_s + '-01-01'
  end

  # TODO: this should be refactored to be collection object centric AFTER
  # it is spec'd
  def self.latest_date(project_id)
    a = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:start_date_year)
    b = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:end_date_year)

    c = Time.now.strftime('%Y-%m-%d')

    return c if a.nil? && b.nil?

    d = nil

    if a && b
      if a > b
        d = a
      end
    else
      d = a || b
    end

    d.to_s + '/12/31'
  end

  # TODO: Clarify this.
  # CAREFULL - this isn't _in_, this is *with*, if it was in it would be spatial query, not a join(:geographic_items)
  #
  # Find all collection objects which have collecting events which have georeferences which have geographic_items which
  # are located within the geographic item supplied
  # @param [GeographicItem] geographic_item_id
  # @return [Scope] of CollectionObject
  def self.in_geographic_item(geographic_item, limit, steps = false)
    geographic_item_id = geographic_item.id
    if steps
      gi = GeographicItem.find(geographic_item_id)
      # find the geographic_items inside gi
      step_1 = GeographicItem.is_contained_by('any', gi) # .pluck(:id)
      # find the georeferences from the geographic_items
      step_2 = step_1.map(&:georeferences).uniq.flatten
      # find the collecting events connected to the georeferences
      step_3 = step_2.map(&:collecting_event).uniq.flatten
      # find the collection objects associated with the collecting events
      step_4 = step_3.map(&:collection_objects).flatten.map(&:id).uniq
      retval = CollectionObject.where(id: step_4.sort)
    else
      retval = CollectionObject.joins(:geographic_items)
        .where(GeographicItem.contained_by_where_sql(geographic_item.id))
        .limit(limit)
        .includes(:data_attributes, :collecting_event)
    end
    retval
  end

  # TODO: deprecate
  def self.selected_column_names
    @selected_column_names = {
      ce: {in: {}, im: {}},
      co: {in: {}, im: {}},
      bc: {in: {}, im: {}}
    } if @selected_column_names.nil?
    @selected_column_names
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collecting events
  # decode which headers to be displayed for collecting events
  def self.ce_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:ce][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:ce][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.ce_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.collecting_event.internal_attributes
      all_import_das   = collection_object.collecting_event.import_attributes
      group            = collection[:ce]
      unless group.nil?
        group.each_key { |type_key|
          group[type_key.to_sym].each_key { |header|
            this_val = nil
            case type_key.to_sym
            when :in
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            when :im
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                  break
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            else
            end
          }
        }
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for collection objects
  # decode which headers to be displayed for collection objects
  def self.co_headers(project_id)
    CollectionObject.selected_column_names
    cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .distinct
      .pluck(:controlled_vocabulary_term_id)
    # add selectable column names (unselected) to the column name list list
    ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
      @selected_column_names[:co][:in][column_name] = {checked: '0'}
    }
    ImportAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
      .pluck(:import_predicate).uniq.sort.each { |column_name|
        @selected_column_names[:co][:im][column_name] = {checked: '0'}
      }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.co_attributes(collection_object, col_defs)
    retval = []; collection = col_defs
    unless collection.nil?
      # for this collection object, gather all the possible data_attributes
      all_internal_das = collection_object.internal_attributes
      all_import_das   = collection_object.import_attributes
      group            = collection[:co]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = nil
              all_internal_das.each { |da|
                if da.predicate.name == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
        unless group.empty?
          unless group[:im].empty?
            group[:im].each_key { |header|
              this_val = nil
              all_import_das.each { |da|
                if da.import_predicate == header
                  this_val = da.value
                end
              }
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Integer] project_id
  # @return [Hash] of column names and types for biocuration classifications
  # decode which headers to be displayed for biocuration classifications
  def self.bc_headers(project_id)
    CollectionObject.selected_column_names
    # add selectable column names (unselected) to the column name list list
    BiocurationClass.where(project_id:).map(&:name).each { |column_name|
      @selected_column_names[:bc][:in][column_name] = {checked: '0'}
    }
    @selected_column_names
  end

  # @param [CollectionObject] collection_object from which to extract attributes
  # @param [Hash] col_defs - collection of selected headers, prefixes, and types
  # @return [Array] of attributes
  # Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object
  def self.bc_attributes(collection_object, col_defs)
    retval = []
    collection = col_defs
    unless collection.nil?
      group = collection[:bc]
      unless group.nil?
        unless group.empty?
          unless group[:in].empty?
            group[:in].each_key { |header|
              this_val = collection_object.biocuration_classes.map(&:name).include?(header) ? '1' : '0'
              retval.push(this_val) # push one value (nil or not) for each selected header
            }
          end
        end
      end
    end
    retval
  end

  # @param [Array] collecting_event_ids (e.g., from CollectingEvent.in_date_range)
  # @param [Array] area_object_ids (e.g., from GeographicItem.gather_selected_data())
  # @return [Scope] of intersection of collecting events (usually by date range)
  #   and collection objects (usually by inclusion in geographic areas/items)
  def self.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id)
    collecting_events_clause = {collecting_event_id: collecting_event_ids, project: project_id}
    area_objects_clause      = {id: area_object_ids, project: project_id}

    if (collecting_event_ids.empty?)
      collecting_events_clause = {project: project_id}
    end

    if (area_object_ids.empty?)
      area_objects_clause = {}
      if (area_set)
        area_objects_clause = 'false'
      end
    end

    retval = CollectionObject.joins(:collecting_event)
      .where(collecting_events_clause)
      .where(area_objects_clause)
    retval
  end

  # TODO: move to filter
  # @param [Hash] search_start_date string in form 'yyyy-mm-dd'
  # @param [Hash] search_end_date string in form 'yyyy-mm-dd'
  # @param [Hash] partial_overlap 'on' or 'off'
  # @return [Scope] of selected collection objects through collecting events with georeferences, remember to scope to project!
  def self.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on')
    allow_partial = (partial_overlap.downcase == 'off' ? false : true) # TODO: Just get the correct values from the form!
    q = Queries::CollectingEvent::Filter.new(start_date: search_start_date, end_date: search_end_date, partial_overlap_dates: allow_partial)
    joins(:collecting_event).where(q.between_date_range_facet.to_sql)
  end

  # @param used_on [String] required, one of `TaxonDetermination`, `BiologicalAssociation`
  # @return [Scope]
  #    the max 10 most recently used collection_objects, as `used_on`
  def self.used_recently(user_id, project_id, used_on = '')
    return [] if used_on != 'TaxonDetermination' && used_on != 'BiologicalAssociation'
    t = case used_on
        when 'TaxonDetermination'
          TaxonDetermination.arel_table
        when 'BiologicalAssociation'
          BiologicalAssociation.arel_table
        end

    p = CollectionObject.arel_table

    # i is a select manager
    i = case used_on
        when 'BiologicalAssociation'
          t.project(t['biological_association_subject_id'], t['updated_at']).from(t)
            .where(
              t['updated_at'].gt(1.week.ago).and(
                t['biological_association_subject_type'].eq('CollectionObject')
              )
            )
              .where(t['updated_by_id'].eq(user_id))
              .where(t['project_id'].eq(project_id))
              .order(t['updated_at'].desc)
        else
          # TODO: update to reference new TaxonDetermination
          t.project(t['taxon_determination_object_id'], t['taxon_determination_object_type'], t['updated_at']).from(t)
            .where(t['updated_at'].gt( 1.week.ago ))
            .where(t['updated_by_id'].eq(user_id))
            .where(t['project_id'].eq(project_id))
            .order(t['updated_at'].desc)
        end

    # z is a table alias
    z = i.as('recent_t')

    j = case used_on
        when 'BiologicalAssociation'
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(
            z['biological_association_subject_id'].eq(p['id'])
          ))
        else
          # TODO: needs to be fixed to scope the taxon_determination_object_type
          Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(z['taxon_determination_object_id'].eq(p['id'])))
        end

    CollectionObject.joins(j).pluck(:id).uniq
  end

  # @params target [String] one of `TaxonDetermination`, `BiologicalAssociation` , nil
  # @return [Hash] otus optimized for user selection
  def self.select_optimized(user_id, project_id, target = nil)
    r = used_recently(user_id, project_id, target)
    h = {
      quick: [],
      pinboard: CollectionObject.pinned_by(user_id).where(project_id:).to_a,
      recent: []
    }

    if target && !r.empty?
      n = target.tableize.to_sym
      h[:recent] = CollectionObject.where('"collection_objects"."id" IN (?)', r.first(10) ).to_a
      h[:quick] = (CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a  +
                   CollectionObject.where('"collection_objects"."id" IN (?)', r.first(4) ).to_a).uniq
    else
      h[:recent] = CollectionObject.where(project_id:, updated_by_id: user_id).order('updated_at DESC').limit(10).to_a
      h[:quick] = CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a
    end

    h
  end

  # TODO: Unify with Extract in concern
  # @return [Identifier::Local::CatalogNumber, nil]
  #   the first (position) catalog number for this collection object, either on specimen, or container
  def preferred_catalog_number
    if i = Identifier::Local::CatalogNumber.where(identifier_object: self).order(:position).first
      i
    else
      if container
        container.identifiers.where(identifiers: {type: 'Identifier::Local::CatalogNumber'}).order(:position).first
      else
        nil
      end
    end
  end

  # @return [Identifier::Local::RecordNumber, nil]
  #   the first (position) record_Number, on a specimen
  #   !1 Doesn't presently support containers
  def preferred_record_number
    Identifier::Local::RecordNumber.where(identifier_object: self).order(:position).first
  end

  def geographic_name_classification
    # don't load the whole object, just the fields we need
    if a = DwcOccurrence.where(dwc_occurrence_object: self).select(:country, :stateProvince, :county).first

      c = a.country
      s = a.stateProvince
      y = a.county

      v = ::Utilities::Geo::DICTIONARY[c]
      c = v if v
      # s = v if v = ::Utilities::Geo::DICTIONARY[s] # None in there yet
      # y = v if v = ::Utilities::Geo::DICTIONARY[y] # None in there yet

      return {
        country: c,
        state: s,
        county: y
      }
    end
  end

  # return [Boolean]
  #    True if instance is a subclass of BiologicalCollectionObject
  def is_biological?
    self.class <= BiologicalCollectionObject ? true : false
  end

  def annotations
    h = annotations_hash
    (h['biocuration classifications'] = biocuration_classes) if is_biological? && biocuration_classifications.load.any?
    h
  end

  def sv_missing_accession_fields
    soft_validations.add(:accessioned_at, 'Date is not selected') if self.accessioned_at.nil? && !self.accession_provider.nil?
    soft_validations.add(:base, 'Provider is not selected') if !self.accessioned_at.nil? && self.accession_provider.nil?
  end

  def sv_missing_deaccession_fields
    soft_validations.add(:deaccessioned_at, 'Date is not selected') if self.deaccessioned_at.nil? && self.deaccession_reason.present?
    soft_validations.add(:base, 'Recipient is not selected') if self.deaccession_recipient.nil? && self.deaccession_reason && self.deaccessioned_at
    soft_validations.add(:deaccession_reason, 'Reason is is not defined') if self.deaccession_reason.blank? && self.deaccession_recipient && self.deaccessioned_at
  end

  def sv_missing_determination
    # see biological_collection_object
  end

  def sv_missing_collecting_event
    # see biological_collection_object
  end

  def sv_missing_preparation_type
    # see biological_collection_object
  end

  def sv_missing_repository
    # WHY? -  see biological_collection_object
  end

  def sv_missing_biocuration_classification
    # see biological_collection_object
  end

  # See Depiction#destroy_image_stub_collection_object
  # Used to determin if the CO can be
  # destroy after moving an image off
  # this object.
  def is_image_stub?
    r = [
      collecting_event_id.blank?,
      !depictions.reload.any?,
      identifiers.count <= 1,
      !taxon_determinations.any?,
      !type_materials.any?,
      !citations.any?,
      !data_attributes.any?,
      !notes.any?,
      !observations.any?
    ]

   !r.include?(false)

  end

  protected

  def collecting_event_belongs_to_project
    if collecting_event&.persisted? && (Current.project_id || project_id)
      errors.add(:base, 'collecting event is not from this project') if collecting_event.project_id != (Current.project_id || project_id)
    end
  end

  def check_that_both_of_category_and_total_are_not_present
    errors.add(:ranged_lot_category_id, 'Both ranged_lot_category and total can not be set') if ranged_lot_category_id.present? && total.present?
  end

  def check_that_either_total_or_ranged_lot_category_id_is_present
    errors.add(:base, 'Either total or a ranged lot category must be provided') if ranged_lot_category_id.blank? && total.blank?
  end

  def assign_type_if_total_or_ranged_lot_category_id_provided
    if self.total == 1
      self.type = 'Specimen'
    elsif self.total.to_i > 1
      self.type = 'Lot'
    elsif total.nil? && ranged_lot_category_id.present?
      self.type = 'RangedLot'
    end
    true
  end

  def reject_collecting_event(attributed)
    reject = true
    CollectingEvent.core_attributes.each do |a|
      if attributed[a].present?
        reject = false
        break
      end
    end
    # !! does not account for georeferences_attributes!
    reject
  end

end

Class Method Details

.batch_update(params) ⇒ Object

# File 'app/models/collection_object.rb', line 191

def self.batch_update(params)
  request = QueryBatchRequest.new(
    async_cutoff: params[:async_cutoff] || 50,
    klass: 'CollectionObject',
    object_filter_params: params[:collection_object_query],
    object_params: params[:collection_object],
    preview: params[:preview],
  )

  request.cap = 1000

  query_batch_update(request)
end

.batch_update_dwc_occurrence(params) ⇒ Object

# File 'app/models/collection_object.rb', line 205

def self.batch_update_dwc_occurrence(params)
  q = Queries::CollectionObject::Filter.new(params).all

  r = BatchResponse.new
  r.method = 'batch_update_dwc_occurrence'
  r.klass = 'CollectionObject'

  c = q.all.count

  if c == 0 || c > 10000
    r.cap_reason = 'Too many (or no) collection objects (max 10k)'
    return r
  end

  if c < 51
    q.each do |co|
      co.set_dwc_occurrence
      r.updated.push co.id
    end
  else
    r.async = true
    q.each do |co|
      co.dwc_occurrence_update_query
    end
  end

  return r
end

.bc_attributes(collection_object, col_defs) ⇒ Array

Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object

Parameters:

• collection_object (CollectionObject) —

  from which to extract attributes

• col_defs (Hash) —

  • collection of selected headers, prefixes, and types

Returns:

• (Array) —

  of attributes

# File 'app/models/collection_object.rb', line 506

def self.bc_attributes(collection_object, col_defs)
  retval = []
  collection = col_defs
  unless collection.nil?
    group = collection[:bc]
    unless group.nil?
      unless group.empty?
        unless group[:in].empty?
          group[:in].each_key { |header|
            this_val = collection_object.biocuration_classes.map(&:name).include?(header) ? '1' : '0'
            retval.push(this_val) # push one value (nil or not) for each selected header
          }
        end
      end
    end
  end
  retval
end

.bc_headers(project_id) ⇒ Hash

decode which headers to be displayed for biocuration classifications

Parameters:

• project_id (Integer)

Returns:

• (Hash) —

  of column names and types for biocuration classifications

# File 'app/models/collection_object.rb', line 493

def self.bc_headers(project_id)
  CollectionObject.selected_column_names
  # add selectable column names (unselected) to the column name list list
  BiocurationClass.where(project_id:).map(&:name).each { |column_name|
    @selected_column_names[:bc][:in][column_name] = {checked: '0'}
  }
  @selected_column_names
end

.breakdown_buffered(collection_objects) ⇒ Hash

Returns a unque list of buffered_ values observed in the collection objects passed.

Returns:

• (Hash) —

  a unque list of buffered_ values observed in the collection objects passed

# File 'app/models/collection_object.rb', line 264

def self.breakdown_buffered(collection_objects)
  collection_objects = [collection_objects] if collection_objects.class != Array
  breakdown = {}
  categories = BUFFERED_ATTRIBUTES

  categories.each do |c|
    breakdown[c] = []
  end

  categories.each do |c|
    collection_objects.each do |co|
      breakdown[c].push co.send(c)
    end
  end

  categories.each do |c|
    breakdown[c].uniq!
  end

  breakdown
end

.breakdown_status(collection_objects) ⇒ Object

TODO: move to a helper

# File 'app/models/collection_object.rb', line 241

def self.breakdown_status(collection_objects)
  collection_objects = [collection_objects] if collection_objects.class != Array

  breakdown = {
    total_objects:     collection_objects.length,
    collecting_events: {},
    determinations:    {},
    bio_overview:      []
  }

  breakdown.merge!(breakdown_buffered(collection_objects))

  collection_objects.each do |co|
    breakdown[:collecting_events].merge!(co => co.collecting_event) if co.collecting_event
    breakdown[:determinations].merge!(co => co.taxon_determinations) if co.taxon_determinations.load.any?
    breakdown[:bio_overview].push([co.total, co.biocuration_classes.collect { |a| a.name }])
  end

  breakdown
end

.ce_attributes(collection_object, col_defs) ⇒ Array

Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object

Parameters:

• collection_object (CollectionObject) —

  from which to extract attributes

• col_defs (Hash) —

  • collection of selected headers, prefixes, and types

Returns:

• (Array) —

  of attributes

# File 'app/models/collection_object.rb', line 391

def self.ce_attributes(collection_object, col_defs)
  retval = []; collection = col_defs
  unless collection.nil?
    # for this collection object, gather all the possible data_attributes
    all_internal_das = collection_object.collecting_event.internal_attributes
    all_import_das   = collection_object.collecting_event.import_attributes
    group            = collection[:ce]
    unless group.nil?
      group.each_key { |type_key|
        group[type_key.to_sym].each_key { |header|
          this_val = nil
          case type_key.to_sym
          when :in
            all_internal_das.each { |da|
              if da.predicate.name == header
                this_val = da.value
                break
              end
            }
            retval.push(this_val) # push one value (nil or not) for each selected header
          when :im
            all_import_das.each { |da|
              if da.import_predicate == header
                this_val = da.value
                break
              end
            }
            retval.push(this_val) # push one value (nil or not) for each selected header
          else
          end
        }
      }
    end
  end
  retval
end

.ce_headers(project_id) ⇒ Hash

decode which headers to be displayed for collecting events

Parameters:

• project_id (Integer)

Returns:

• (Hash) —

  of column names and types for collecting events

# File 'app/models/collection_object.rb', line 371

def self.ce_headers(project_id)
  CollectionObject.selected_column_names
  cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
    .distinct
    .pluck(:controlled_vocabulary_term_id)
  # add selectable column names (unselected) to the column name list list
  ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
    @selected_column_names[:ce][:in][column_name] = {checked: '0'}
  }
  ImportAttribute.where(project_id:, attribute_subject_type: 'CollectingEvent')
    .pluck(:import_predicate).uniq.sort.each { |column_name|
      @selected_column_names[:ce][:im][column_name] = {checked: '0'}
    }
  @selected_column_names
end

.co_attributes(collection_object, col_defs) ⇒ Array

Retrieve all the attributes associated with the column names (col_defs) for a specific collection_object

Parameters:

• collection_object (CollectionObject) —

  from which to extract attributes

• col_defs (Hash) —

  • collection of selected headers, prefixes, and types

Returns:

• (Array) —

  of attributes

# File 'app/models/collection_object.rb', line 451

def self.co_attributes(collection_object, col_defs)
  retval = []; collection = col_defs
  unless collection.nil?
    # for this collection object, gather all the possible data_attributes
    all_internal_das = collection_object.internal_attributes
    all_import_das   = collection_object.import_attributes
    group            = collection[:co]
    unless group.nil?
      unless group.empty?
        unless group[:in].empty?
          group[:in].each_key { |header|
            this_val = nil
            all_internal_das.each { |da|
              if da.predicate.name == header
                this_val = da.value
              end
            }
            retval.push(this_val) # push one value (nil or not) for each selected header
          }
        end
      end
      unless group.empty?
        unless group[:im].empty?
          group[:im].each_key { |header|
            this_val = nil
            all_import_das.each { |da|
              if da.import_predicate == header
                this_val = da.value
              end
            }
            retval.push(this_val) # push one value (nil or not) for each selected header
          }
        end
      end
    end
  end
  retval
end

.co_headers(project_id) ⇒ Hash

decode which headers to be displayed for collection objects

Parameters:

• project_id (Integer)

Returns:

• (Hash) —

  of column names and types for collection objects

# File 'app/models/collection_object.rb', line 431

def self.co_headers(project_id)
  CollectionObject.selected_column_names
  cvt_list = InternalAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
    .distinct
    .pluck(:controlled_vocabulary_term_id)
  # add selectable column names (unselected) to the column name list list
  ControlledVocabularyTerm.where(id: cvt_list).map(&:name).sort.each { |column_name|
    @selected_column_names[:co][:in][column_name] = {checked: '0'}
  }
  ImportAttribute.where(project_id:, attribute_subject_type: 'CollectionObject')
    .pluck(:import_predicate).uniq.sort.each { |column_name|
      @selected_column_names[:co][:im][column_name] = {checked: '0'}
    }
  @selected_column_names
end

.earliest_date(project_id) ⇒ Object

TODO: this should be refactored to be collection object centric AFTER it is spec’d

# File 'app/models/collection_object.rb', line 288

def self.earliest_date(project_id)
  a = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:start_date_year)
  b = CollectingEvent.joins(:collection_objects).where(project_id:).minimum(:end_date_year)

  return EARLIEST_DATE if a.nil? && b.nil?  # 1700-01-01

  d = nil

  if a && b
    if a < b
      d = a
    end
  else
    d = a || b
  end
  d.to_s + '-01-01'
end

.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id) ⇒ Scope

Returns of intersection of collecting events (usually by date range) and collection objects (usually by inclusion in geographic areas/items).

Parameters:

• collecting_event_ids (Array) —

  (e.g., from CollectingEvent.in_date_range)

• area_object_ids (Array) —

  (e.g., from GeographicItem.gather_selected_data())

Returns:

• (Scope) —

  of intersection of collecting events (usually by date range) and collection objects (usually by inclusion in geographic areas/items)

# File 'app/models/collection_object.rb', line 529

def self.from_collecting_events(collecting_event_ids, area_object_ids, area_set, project_id)
  collecting_events_clause = {collecting_event_id: collecting_event_ids, project: project_id}
  area_objects_clause      = {id: area_object_ids, project: project_id}

  if (collecting_event_ids.empty?)
    collecting_events_clause = {project: project_id}
  end

  if (area_object_ids.empty?)
    area_objects_clause = {}
    if (area_set)
      area_objects_clause = 'false'
    end
  end

  retval = CollectionObject.joins(:collecting_event)
    .where(collecting_events_clause)
    .where(area_objects_clause)
  retval
end

.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on') ⇒ Scope

TODO: move to filter

Parameters:

• search_start_date (Hash) (defaults to: nil) —

  string in form ‘yyyy-mm-dd’

• search_end_date (Hash) (defaults to: nil) —

  string in form ‘yyyy-mm-dd’

• partial_overlap (Hash) (defaults to: 'on') —

  ‘on’ or ‘off’

Returns:

• (Scope) —

  of selected collection objects through collecting events with georeferences, remember to scope to project!

# File 'app/models/collection_object.rb', line 555

def self.in_date_range(search_start_date: nil, search_end_date: nil, partial_overlap: 'on')
  allow_partial = (partial_overlap.downcase == 'off' ? false : true) # TODO: Just get the correct values from the form!
  q = Queries::CollectingEvent::Filter.new(start_date: search_start_date, end_date: search_end_date, partial_overlap_dates: allow_partial)
  joins(:collecting_event).where(q.between_date_range_facet.to_sql)
end

.in_geographic_item(geographic_item, limit, steps = false) ⇒ Scope

TODO: Clarify this. CAREFULL - this isn’t in, this is with, if it was in it would be spatial query, not a join(:geographic_items)

Find all collection objects which have collecting events which have georeferences which have geographic_items which are located within the geographic item supplied

Parameters:

• geographic_item_id (GeographicItem)

Returns:

• (Scope) —

  of CollectionObject

# File 'app/models/collection_object.rb', line 336

def self.in_geographic_item(geographic_item, limit, steps = false)
  geographic_item_id = geographic_item.id
  if steps
    gi = GeographicItem.find(geographic_item_id)
    # find the geographic_items inside gi
    step_1 = GeographicItem.is_contained_by('any', gi) # .pluck(:id)
    # find the georeferences from the geographic_items
    step_2 = step_1.map(&:georeferences).uniq.flatten
    # find the collecting events connected to the georeferences
    step_3 = step_2.map(&:collecting_event).uniq.flatten
    # find the collection objects associated with the collecting events
    step_4 = step_3.map(&:collection_objects).flatten.map(&:id).uniq
    retval = CollectionObject.where(id: step_4.sort)
  else
    retval = CollectionObject.joins(:geographic_items)
      .where(GeographicItem.contained_by_where_sql(geographic_item.id))
      .limit(limit)
      .includes(:data_attributes, :collecting_event)
  end
  retval
end

.latest_date(project_id) ⇒ Object

TODO: this should be refactored to be collection object centric AFTER it is spec’d

# File 'app/models/collection_object.rb', line 308

def self.latest_date(project_id)
  a = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:start_date_year)
  b = CollectingEvent.joins(:collection_objects).where(project_id:).maximum(:end_date_year)

  c = Time.now.strftime('%Y-%m-%d')

  return c if a.nil? && b.nil?

  d = nil

  if a && b
    if a > b
      d = a
    end
  else
    d = a || b
  end

  d.to_s + '/12/31'
end

.select_optimized(user_id, project_id, target = nil) ⇒ Hash

Returns otus optimized for user selection.

Returns:

• (Hash) —

  otus optimized for user selection

# File 'app/models/collection_object.rb', line 614

def self.select_optimized(user_id, project_id, target = nil)
  r = used_recently(user_id, project_id, target)
  h = {
    quick: [],
    pinboard: CollectionObject.pinned_by(user_id).where(project_id:).to_a,
    recent: []
  }

  if target && !r.empty?
    n = target.tableize.to_sym
    h[:recent] = CollectionObject.where('"collection_objects"."id" IN (?)', r.first(10) ).to_a
    h[:quick] = (CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a  +
                 CollectionObject.where('"collection_objects"."id" IN (?)', r.first(4) ).to_a).uniq
  else
    h[:recent] = CollectionObject.where(project_id:, updated_by_id: user_id).order('updated_at DESC').limit(10).to_a
    h[:quick] = CollectionObject.pinned_by(user_id).pinboard_inserted.where(project_id:).to_a
  end

  h
end

.selected_column_names ⇒ Object

TODO: deprecate

# File 'app/models/collection_object.rb', line 359

def self.selected_column_names
  @selected_column_names = {
    ce: {in: {}, im: {}},
    co: {in: {}, im: {}},
    bc: {in: {}, im: {}}
  } if @selected_column_names.nil?
  @selected_column_names
end

.sequence_join_hack_sql ⇒ Object

This is a hack, maybe related to a Rails 5.1 bug. It returns the SQL that works in 5.0/4.2 that links CollectionObject to Sequences: joins(derived_extracts: [:derived_sequences])

# File 'app/models/collection_object.rb', line 176

def self.sequence_join_hack_sql
  %Q{INNER JOIN  "origin_relationships"
             ON  "origin_relationships"."old_object_id" = "collection_objects"."id"
                AND  "origin_relationships"."new_object_type" = 'Extract'
                AND  "origin_relationships"."old_object_type" = 'CollectionObject'
     INNER JOIN  "extracts"
             ON  "extracts"."id" =  "origin_relationships"."new_object_id"
     INNER JOIN  "origin_relationships" "origin_relationships_extracts_join"
             ON  "origin_relationships_extracts_join"."old_object_id" = "extracts"."id"
                AND  "origin_relationships_extracts_join"."new_object_type" = 'Sequence'
                AND  "origin_relationships_extracts_join"."old_object_type" = 'Extract'
     INNER JOIN  "sequences"
             ON  "sequences"."id" = "origin_relationships_extracts_join"."new_object_id"}
end

.used_recently(user_id, project_id, used_on = '') ⇒ Scope

Returns the max 10 most recently used collection_objects, as ‘used_on`.

Parameters:

• used_on (String) (defaults to: '') —

  required, one of ‘TaxonDetermination`, `BiologicalAssociation`

Returns:

• (Scope) —

  the max 10 most recently used collection_objects, as ‘used_on`

# File 'app/models/collection_object.rb', line 564

def self.used_recently(user_id, project_id, used_on = '')
  return [] if used_on != 'TaxonDetermination' && used_on != 'BiologicalAssociation'
  t = case used_on
      when 'TaxonDetermination'
        TaxonDetermination.arel_table
      when 'BiologicalAssociation'
        BiologicalAssociation.arel_table
      end

  p = CollectionObject.arel_table

  # i is a select manager
  i = case used_on
      when 'BiologicalAssociation'
        t.project(t['biological_association_subject_id'], t['updated_at']).from(t)
          .where(
            t['updated_at'].gt(1.week.ago).and(
              t['biological_association_subject_type'].eq('CollectionObject')
            )
          )
            .where(t['updated_by_id'].eq(user_id))
            .where(t['project_id'].eq(project_id))
            .order(t['updated_at'].desc)
      else
        # TODO: update to reference new TaxonDetermination
        t.project(t['taxon_determination_object_id'], t['taxon_determination_object_type'], t['updated_at']).from(t)
          .where(t['updated_at'].gt( 1.week.ago ))
          .where(t['updated_by_id'].eq(user_id))
          .where(t['project_id'].eq(project_id))
          .order(t['updated_at'].desc)
      end

  # z is a table alias
  z = i.as('recent_t')

  j = case used_on
      when 'BiologicalAssociation'
        Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(
          z['biological_association_subject_id'].eq(p['id'])
        ))
      else
        # TODO: needs to be fixed to scope the taxon_determination_object_type
        Arel::Nodes::InnerJoin.new(z, Arel::Nodes::On.new(z['taxon_determination_object_id'].eq(p['id'])))
      end

  CollectionObject.joins(j).pluck(:id).uniq
end

Instance Method Details

#annotations ⇒ Object

# File 'app/models/collection_object.rb', line 684

def annotations
  h = annotations_hash
  (h['biocuration classifications'] = biocuration_classes) if is_biological? && biocuration_classifications.load.any?
  h
end

#assign_type_if_total_or_ranged_lot_category_id_provided ⇒ Object (protected)

# File 'app/models/collection_object.rb', line 758

def assign_type_if_total_or_ranged_lot_category_id_provided
  if self.total == 1
    self.type = 'Specimen'
  elsif self.total.to_i > 1
    self.type = 'Lot'
  elsif total.nil? && ranged_lot_category_id.present?
    self.type = 'RangedLot'
  end
  true
end

#check_that_both_of_category_and_total_are_not_present ⇒ Object (protected)

# File 'app/models/collection_object.rb', line 750

def check_that_both_of_category_and_total_are_not_present
  errors.add(:ranged_lot_category_id, 'Both ranged_lot_category and total can not be set') if ranged_lot_category_id.present? && total.present?
end

#check_that_either_total_or_ranged_lot_category_id_is_present ⇒ Object (protected)

# File 'app/models/collection_object.rb', line 754

def check_that_either_total_or_ranged_lot_category_id_is_present
  errors.add(:base, 'Either total or a ranged lot category must be provided') if ranged_lot_category_id.blank? && total.blank?
end

#collecting_event_belongs_to_project ⇒ Object (protected)

# File 'app/models/collection_object.rb', line 744

def collecting_event_belongs_to_project
  if collecting_event&.persisted? && (Current.project_id || project_id)
    errors.add(:base, 'collecting event is not from this project') if collecting_event.project_id != (Current.project_id || project_id)
  end
end

#dwc_occurrence_update_query ⇒ Object

# File 'app/models/collection_object.rb', line 234

def dwc_occurrence_update_query
  self.send(:set_dwc_occurrence)
end

#geographic_name_classification ⇒ Object

# File 'app/models/collection_object.rb', line 657

def geographic_name_classification
  # don't load the whole object, just the fields we need
  if a = DwcOccurrence.where(dwc_occurrence_object: self).select(:country, :stateProvince, :county).first

    c = a.country
    s = a.stateProvince
    y = a.county

    v = ::Utilities::Geo::DICTIONARY[c]
    c = v if v
    # s = v if v = ::Utilities::Geo::DICTIONARY[s] # None in there yet
    # y = v if v = ::Utilities::Geo::DICTIONARY[y] # None in there yet

    return {
      country: c,
      state: s,
      county: y
    }
  end
end

#is_biological? ⇒ Boolean

return [Boolean]

True if instance is a subclass of BiologicalCollectionObject

Returns:

• (Boolean)

# File 'app/models/collection_object.rb', line 680

def is_biological?
  self.class <= BiologicalCollectionObject ? true : false
end

#is_image_stub? ⇒ Boolean

See Depiction#destroy_image_stub_collection_object Used to determin if the CO can be destroy after moving an image off this object.

Returns:

• (Boolean)

# File 'app/models/collection_object.rb', line 725

def is_image_stub?
  r = [
    collecting_event_id.blank?,
    !depictions.reload.any?,
    identifiers.count <= 1,
    !taxon_determinations.any?,
    !type_materials.any?,
    !citations.any?,
    !data_attributes.any?,
    !notes.any?,
    !observations.any?
  ]

 !r.include?(false)

end

#preferred_catalog_number ⇒ Identifier::Local::CatalogNumber^?

TODO: Unify with Extract in concern

Returns:

• (Identifier::Local::CatalogNumber, nil) —

  the first (position) catalog number for this collection object, either on specimen, or container

# File 'app/models/collection_object.rb', line 638

def preferred_catalog_number
  if i = Identifier::Local::CatalogNumber.where(identifier_object: self).order(:position).first
    i
  else
    if container
      container.identifiers.where(identifiers: {type: 'Identifier::Local::CatalogNumber'}).order(:position).first
    else
      nil
    end
  end
end

#preferred_record_number ⇒ Identifier::Local::RecordNumber^?

Returns the first (position) record_Number, on a specimen !1 Doesn’t presently support containers.

Returns:

• (Identifier::Local::RecordNumber, nil) —

  the first (position) record_Number, on a specimen !1 Doesn’t presently support containers

# File 'app/models/collection_object.rb', line 653

def preferred_record_number
  Identifier::Local::RecordNumber.where(identifier_object: self).order(:position).first
end

#reject_collecting_event(attributed) ⇒ Object (protected)

# File 'app/models/collection_object.rb', line 769

def reject_collecting_event(attributed)
  reject = true
  CollectingEvent.core_attributes.each do |a|
    if attributed[a].present?
      reject = false
      break
    end
  end
  # !! does not account for georeferences_attributes!
  reject
end

#sv_missing_accession_fields ⇒ Object

# File 'app/models/collection_object.rb', line 690

def sv_missing_accession_fields
  soft_validations.add(:accessioned_at, 'Date is not selected') if self.accessioned_at.nil? && !self.accession_provider.nil?
  soft_validations.add(:base, 'Provider is not selected') if !self.accessioned_at.nil? && self.accession_provider.nil?
end

#sv_missing_biocuration_classification ⇒ Object

# File 'app/models/collection_object.rb', line 717

def sv_missing_biocuration_classification
  # see biological_collection_object
end

#sv_missing_collecting_event ⇒ Object

# File 'app/models/collection_object.rb', line 705

def sv_missing_collecting_event
  # see biological_collection_object
end

#sv_missing_deaccession_fields ⇒ Object

# File 'app/models/collection_object.rb', line 695

def sv_missing_deaccession_fields
  soft_validations.add(:deaccessioned_at, 'Date is not selected') if self.deaccessioned_at.nil? && self.deaccession_reason.present?
  soft_validations.add(:base, 'Recipient is not selected') if self.deaccession_recipient.nil? && self.deaccession_reason && self.deaccessioned_at
  soft_validations.add(:deaccession_reason, 'Reason is is not defined') if self.deaccession_reason.blank? && self.deaccession_recipient && self.deaccessioned_at
end

#sv_missing_determination ⇒ Object

# File 'app/models/collection_object.rb', line 701

def sv_missing_determination
  # see biological_collection_object
end

#sv_missing_preparation_type ⇒ Object

# File 'app/models/collection_object.rb', line 709

def sv_missing_preparation_type
  # see biological_collection_object
end

#sv_missing_repository ⇒ Object

# File 'app/models/collection_object.rb', line 713

def sv_missing_repository
  # WHY? -  see biological_collection_object
end