Changelog

master (unreleased)

New Features

Changes

Bugs Fixed

  • #964: Fix delayed_sidekiq worker to handle UUID primary keys correctly.

8.0.0-beta (2024-08-27)

New Features

  • #962: ElasticSearch v.8 support added

  • delete_all_enabled setting introduced to align Chewy.massacre with wildcard indices deletion disabled in ES 8 by default

Changes

Bugs Fixed

7.6.0 (2024-05-03)

Changes

  • #933: Relax allowed elasticsearch dependency versions. ([@mjankowski][])

Bugs Fixed

  • #937: Fix for race condition while using the delayed_sidekiq strategy. Also, fix for Redis bloating in case of reindexing error (@skcc321)

  • #947: Fix intermittent time-based failure in delayed sidekiq spec. ([@mjankowski][])

7.5.1 (2024-01-30)

New Features

  • #925: Add configuration option for default scope cleanup behavior. (@barthez)

Changes

Bugs Fixed

7.5.0 (2024-01-15)

New Features

7.4.0 (2023-12-13)

New Features

Changes

Bugs Fixed

7.3.6 (2023-12-13)

New Features

Changes

Bugs Fixed

7.3.5 (2023-12-06)

New Features

Changes

Bugs Fixed

7.3.4 (2023-08-29)

New Features

Changes

Bugs Fixed

7.3.3 (2023-07-07)

New Features

Changes

Bugs Fixed

7.3.2 (2023-04-20)

New Features

Changes

Bugs Fixed

  • #861: Fix bug in mock_elasticsearch_response_sources (@lafeber)

7.3.1 (2023-04-20)

Bugs Fixed

  • #874: Fix chewy:journal:clean task for ruby 3.x. (@muk-ai)
  • #882: Fix memory leak during chewy:reset for ruby 3.2 (@konalegi)

7.3.0 (2023-04-03)

New Features

  • #869: New strategy - delayed_sidekiq. Allow passing strategy: :delayed_sidekiq option to SomeIndex.import([1, ...], strategy: :delayed_sidekiq). The strategy is compatible with update_fields option as well. ([@skcc321][])
  • #879: Configure CI to check for ruby 3.2 compatibility. (@konalegi)

Changes

Bugs Fixed

  • #856: Fix return value of subscribed_task_stats used in rake tasks. ([@fabiormoura][])

7.2.7 (2022-11-15)

New Features

  • #857: Allow passing wait_for_completion, request_per_second and scroll_size options to chewy:journal:clean rake task and delete_all query builder method. (@konalegi)(@barthez)

Changes

Bugs Fixed

  • #863: Fix crutches call doesn't respect update_fields option. ([@skcc321][])

7.2.6 (2022-06-13)

New Features

Bugs Fixed

7.2.5 (2022-03-04)

New Features

  • #827: Add :lazy_sidekiq strategy, that defers not only importing but also update_index callback evaluation for created and updated objects. ([@sl4vr][])
  • #827: Add :atomic_no_refresh strategy. Like :atomic, but refresh=false parameter is set. (@barthez)
  • #827: Add :no_refresh chain call to update_index matcher to ensure import was called with refresh=false. (@barthez)

Bugs Fixed

7.2.4 (2022-02-03)

New Features

Bugs Fixed

7.2.3 (2021-10-29)

New Features

Changes

  • #817: Show warning message during rake chewy:reset or chewy:upgrade if journaling is disabled. (@konalegi)

Bugs Fixed

7.2.2 (2021-05-24)

Changes

  • #800: Revert #787 progressbar feature to avoid performance degradation in parallel import (@rabotyaga)

  • #795: (Breaking) Change the Chewy::Search::Parameters::Order implementation to use Array (@jiajiawang):

    • To allow multiple sorting options that may have the same key name. For example script based sorting whose key will always be _script.
    • Behaviour change of chained order calls.
      • e.g. .order(_script: {a: 1}).order(_script: {b: 2})
      • Before {:sort=>[{"_script"=>{:b=>2}}]}
      • After {:sort=>[{"_script"=>{:a=>1}},{"_script"=>{:b=>2}}]}

  • #654: Add helpers and matchers for testing (@Vitalina-Vakulchyk):

    • mock_elasticsearch_response helpers both Rspec and Minitest - to mock elasticsearch response
    • mock_elasticsearch_response_sources helpers both Rspec and Minitest - to mock elasticsearch response sources
    • assert_elasticsearch_query helper for Minitest - to compare request and expected query (returns true/false)
    • build_query matcher for Rspec - to compare request and expected query (returns true/false)

7.2.1 (2021-05-11)

New Features

  • #469: Add ability to output the progressbar with ENV['PROGRESS'] during reset rake tasks (@Vitalina-Vakulchyk):
    • for rake chewy:reset and rake chewy:parallel:reset
    • progressbar is hidden by default, set ENV['PROGRESS'] to true to display it

Bugs Fixed

  • #796: Fix clear scroll: pass scroll_id in body, as passing in path parameters is deprecated and can overflow http.max_initial_line_length (@rabotyaga)

7.0.1 (2021-05-03)

Changes

  • #792: Skip ES version memoization for search requests (@rabotyaga)
    • See the Migration Guide for details

7.2.0 (2021-04-19)

New Features

  • #778: Add ignore_blank option to field method (@Vitalina-Vakulchyk):
    • true by default for the geo_point type
    • false by default for other types

Changes

  • #783: (Breaking) Remove Chewy::Type, simplify DSL (@rabotyaga)
    • Remove the Chewy::Type class
      • e.g. remove CitiesIndex::City / CitiesIndex.city
        • CitiesIndex::City.import! ... becomes CitiesIndex.import! ...
    • Simplify index DSL:
      • define_type block -> index_scope clause
        • it can be omitted completely, if you don't need to specify the scope or options, e.g. name
    • Remove type names from string representations:
      • in update_index ActiveRecord helper and RSpec matcher, e.g.
        • update_index('cities#city') -> update_index('cities')
        • update_index(UsersIndex::User) -> update_index(UsersIndex)
      • in rake tasks (e.g. rake chewy:update[cities#city] -> rake chewy:update[cities])
      • in rake tasks output (e.g. Imported CitiesIndex::City in 1s, stats: index 3 -> Imported CitiesIndex in 1s, stats: index 3)
    • Use index name instead of type name in loader additional scope
      • e.g. CitiesIndex.filter(...).load(city: {scope: City.where(...)}) -> CitiesIndex.filter(...).load(cities: {scope: City.where(...)})
  • #692: Add .update_mapping to Index class (@Vitalina-Vakulchyk):
    • Wrapped Elasticsearch gem .put_mapping with .update_mapping in Index class
    • Add rake chewy:update_mapping task
  • #594: Add .reindex to Index class (@Vitalina-Vakulchyk):
    • Wrapped Elasticsearch gem .reindex with .reindex in Index class
    • Add rake chewy:reindex task
  • #679: Wrapped Elasticsearch::API::Indices::Actions#clear_cache with .clear_cache in Index class (@Vitalina-Vakulchyk)
  • #495: Ability to change Rails console strategy with Chewy.console_strategy (@Vitalina-Vakulchyk)
  • #778: (Breaking) Drop support for Ruby 2.5 (@Vitalina-Vakulchyk)
  • #776: (Breaking) Removal of unnecessary features and integrations (@Vitalina-Vakulchyk):
    • aws-sdk-sqs / shoryuken
    • mongoid
    • sequel
    • will_paginate
    • resque
  • #769: (Breaking) Removal of deprecated methods and rake tasks (@Vitalina-Vakulchyk):
    • Chewy::Index.index_params is removed, use Chewy::Index.specification_hash instead
    • Chewy::Index.derivable_index_name is removed, use Chewy::Index.derivable_name instead
    • Chewy::Index.default_prefix is removed, use Chewy::Index.prefix instead
    • Chewy::Index.build_index_name is removed, use Chewy::Index.index_name instead
    • Chewy::RakeHelper.reset_index is removed, use Chewy::RakeHelper.reset instead
    • Chewy::RakeHelper.reset_all is removed, use Chewy::RakeHelper.reset instead
    • Chewy::RakeHelper.update_index is removed, use Chewy::RakeHelper.update instead
    • Chewy::RakeHelper.update_all is removed, use Chewy::RakeHelper.update instead
    • rake chewy:apply_changes_from is removed, use rake chewy:journal:apply instead
    • rake chewy:clean_journal is removed, use rake chewy:journal:clean instead

7.1.0 (2021-03-03)

Changes

Bugs Fixed

  • #722: Remove alias_method_chain, use Module#prepend instead (@dalthon)

7.0.0 (2021-02-22)

New Features

Changes

  • #757: (Breaking) Fix Chewy::Index.index & Chewy::Index.aliases to correctly report indexes and aliases (@mpeychich, @dalthon)
  • #761: Avoid fetching scope data to check if it is blank (@dalthon)

6.0.0 (2021-02-11)

Changes

Bugs Fixed

5.2.0 (2021-01-28)

Changes

  • #734: Add support for Ruby 3. (@lowang)
  • #735: Correct deprecation warning for Elasticsearch 5.6 to 6: empty query for_delete_by_query, delete by alias, index_already_exists_exception renaming. (@bhacaz)
  • #733: Update gemspec dependencies for Rails. Update CI gemfiles and matrix to tests against current LTS Rails versions. (@bhacaz)
  • Tweak some wording and formatting; add a note about compatibility; update copyright; remove broken logo; update the CI badge. (@bbatsov)
  • #714: Update instructions for AWS ElasticSearch. (@olancheg)
  • #728: Fix more ruby 2.7 keyword params deprecation warnings. (@aglushkov)
  • #715: Fixed all deprecation warnings in Ruby 2.7. (@gseddon)
  • #718: Added Ruby 2.7 to CircleCI config. (@mrzasa)
  • #707: Allow configuration of Active Job queue name. (@mrzasa)
  • #711: Setup CI on CircleCI. (@mrzasa)
  • #710: Fix deprecation warning for constructing new BigDecimal. (@AlexVPopov)

5.1.0 (2019-09-24)

Changes

  • #657: (Breaking) Add support for multiple indices in request. (@pyromaniac)
  • #647: (Breaking) Support search_type, request_cache, and allow_partial_search_results as query string parameters. (@mattzollinhofer)
  • #606: Speed up imports when bulk_size is specified. (@yahooguntu)
  • #682: Insert RequestStrategy middleware before ActionDispatch::ShowExceptions. (@dck)

5.0.0 (2018-02-13)

Changes

  • (Breaking) Align the gem version with the most recent ElasticSearch version we support.
  • (Breaking) Chewy.default_field_type is text now.
  • (Breaking) Chewy::Stash was split onto two indexes - Chewy::Stash::Specification and Chewy::Stash::Journal.
  • (Breaking) Data for journal and specification is stored in binary fields base64-encoded to bypass the limits of other fields.
  • (Breaking) #626: Don't underscore suggested index name. (@dm1try)
  • #598: pipeline import option support. (@eManPrague)
  • #625: Proper Rails check. (@nattfodd)
  • #623: Bypass strategy performance improvements. (@DNNX)
  • #620: Avoid index update calls for empty data. (@robertasg)
  • Do not underscore suggested index name on Chewy::Index.index_name call.
  • It is possible now to call root method several times inside a single type definition, the options will be merged. Also, the block isn't required anymore.
  • #565: Fixed some Sequel deprecation warnings. (@arturtr)
  • #577: Fixed some Sequel deprecation warnings. (@matchbookmac)

Bugs Fixed

0.10.1

Changes

  • #558: Improved parallel worker titles

Bugs Fixed

  • #557: Fixed request strategy initial debug message
  • #556: Fixed will objects paginated array initialization when pagination was not used
  • #555: Fixed fields symbol/string value
  • #554: Fixed root field value proc

0.10.0

Breaking changes

  • Changed behavior of Chewy::Index.index_name, it doesn't cache the values anymore.
  • Journal interfaces, related code and rake tasks were completely refactored and are not compatible with the previous version.

Changes

  • #543: Less noisy strategies logging (@Borzik)
  • Parallel import and the corresponding rake tasks.
  • #532: :shoryuken async strategy (@josephchoe)
  • Deprecate Chewy::Index.build_index_name.
  • Rename Chewy::Index.default_prefix to Chewy::Index.prefix. The old one is deprecated.
  • Add Chewy::Type.derivable_name for consistency.
  • Rename Chewy::Index.derivable_index_name to Chewy::Index.derivable_name. Chewy::Index.derivable_index_name and Chewy::Type.derivable_index_name are deprecated.
  • Use normal YAML loading, for the config, we don't need the safe one.
  • #526: default_root_options option (@barthez)
  • Partial indexing ability: it is possible to update only specified fields.
  • New cool rake chewy:deploy task.
  • Selective reset (resets only if necessary): rake chewy:upgrade.
  • Consistency checks and synchronization: rake chewy:sync.
  • Brand new request DSL. Supports ElasticSearch 2 and 5, better usability, architecture and docs.
  • Add Kaminari 1.0 support.
  • #483: skip_index_creation_on_import option (@sergey-kintsel)
  • #481: Ability to use procs for settings (@parallel588)
  • #467: Bulk indexing optimizations with new additional options (@eproulx-petalmd)
  • #438: Configurable sidekiq options (@averell23)

0.9.0

Changes

Bugs Fixed

0.8.4

Changes

  • Brand new import :bulk_size option, set desired ElasticSearch bulk size in bytes
  • Witchcraft™ technology
  • #341: Configurable per-type default import options (@barthez)
  • Various codebase optimizations (@DNNX, @pyromaniac)
  • update_index Rspec matcher messages improvements
  • :all rake tasks deprecation
  • #335: Scoped notification subscriptions in rake tasks (@0x0badc0de)
  • #321: Async strategies workers accept options (@dnd)
  • #314: Prefix is configurable per-index (@mikeyhogarth)
  • #302, #339: Ability to pass proc for transport configuration (@feymartynov, @reidab)
  • #297: ElasticSearch 2 support (@sergeygaychuk)
  • Accessing types with methods is deprecated. Use MyIndex::MyType constant reference instead of MyIndex.my_type method.
  • #294: Sequel adapter improvements (@mrbrdo)

Bugs Fixed

0.8.3

Breaking changes:

  • Chewy.atomic and Chewy.urgent_update= methods was removed from the codebase, use Chewy.strategy block instead.
  • delete_from_index? hook is removed from the codebase.

Changes

  • Sequel support completely reworked to use common ORM implementations + better sequel specs covarage.

Bugs Fixed

  • Sequel objects transactional destruction fix
  • Correct Rspec mocking framework checking (@mainameiz)
  • Atomic strategy is now compatible with custom ids proc.
  • Safe unsubscribe on import ([@marshall-lee][])
  • Correct custom assets path silencer (@davekaro)

0.8.2

Changes

Bugs Fixed

  • Fixed transport logger and tracer configuration

0.8.1

Bugs Fixed

  • Added support of elasticsearch-ruby 1.0.10

0.8.0

Breaking changes:

  • :atomic and :urgent strategies are using import! method raising exceptions

Changes

  • Crutches™ technology
  • Added .script_fields chainable method to query (@ka8725)
  • update_index matcher mocha support (@lardawge)
  • :resque async strategy
  • :sidekiq async strategy (inspired by @sharkzp)
  • Added Query#search_type for search_type request option setup ([@marshall-lee][])

Bugs Fixed

  • Rails 4.2 migrations are not raising UndefinedUpdateStrategy anymore on data updates
  • Mongoid random failing specs fixes ([@marshall-lee][])

0.7.0

Breaking changes:

  • Chewy.use_after_commit_callbacks = false returns previous RDBMS behavior in tests
  • ActiveRecord import is now called after_commit instead of after_save and after_destroy
  • Import now respects default scope and removes unmatched documents
  • delete_from_index? method is deprecated, use ruby define_type User, delete_if: ->{ removed? } do ... end
  • Chewy.request_strategy to configure action controller's request wrapping strategy
  • Chewy.root_strategy to configure the first strategy in stack
  • Default strategy for controller actions is :atomic
  • Default strategy for activerecord migrations is :bypass
  • Default strategy for sandbox console is :bypass
  • Default strategy for rails console is :urgent
  • Chewy.configuration was renamed to Chewy.settings
  • Reworked index update strategies implementation. Chewy.atomic and Chewy.urgent_update are now deprecated in favour of the new Chewy.strategy API.

  • Loading objects for object-sourced types using wrap method is deprecated, load_one method should be used instead. Or method name might be passed to define_type:

      class GeoData
    def self.get_data(elasticsearch_document)
      REDIS.get("geo_data_#{elasticsearch_document.id}")
    end
  end

  ...
    define_type GeoData, load_one_method: :get_data do
      ...
    end

Changes

  • Multiple enhancements by @DNNX
  • Added script_fields to search criteria (@ka8725)

  • ORM adapters now completely relies on the default scope. This means every scope or objects passed to import are merged with default scope so basically there is no need to define delete_if block. Default scope strongly restricts objects which may land in the current index.

      define_type Country.where("rating > 30") do

  end

  # this code would import only countries with rating between 30 and 50
  CountriesIndex::Country.import(Country.where("rating < 50"))

  # the same is true for arrays of objects or ids
  CountriesIndex::Country.import(Country.where("rating < 50").to_a)
  CountriesIndex::Country.import(Country.where("rating < 50").pluck(:id))

  • Object adapter supports custom initial import and load methods, so it could be configured to be used with procs or any class responding to call method.

      class GeoData
    def self.call
      REDIS.get_all
    end
  end

  ...
    define_type GeoData do
      ...
    end

  • Nested fields value procs additional arguments: parent objects.

      define_type Country do
    field :name
    field :cities do
      field :district, value: ->(city, country) { city.districts if country.main? }
    end
  end

  • Implemented basic named scopes

Bugs Fixed

  • script_score allow options (@joeljunstrom)
  • Chewy indexes eaged loading fixes (@leemhenson)
  • Chewy::Index.import nil imports nothing instead of initial data

0.6.2

Changes

Bugs Fixed

  • Removed decay function defaults (@Linuus)
  • Correct config file handling in case of empty file

0.6.1

Changes

  • min_score query option support (@jshirley)
  • Chewy::Query#find method for finding documents by id

0.6.0

Changes

  • Mongoid support YaY! (@fabiotomio, @leemhenson)
  • urgent: true option for update_index is deprecated and will be removed soon, use Chewy.atomic instead
  • timeout and timed_out support (@MarkMurphy)
  • will_paginate support (@josecoelho)

Bugs Fixed

  • All the query chainable methods delegated to indexes and types (partially @Linuus)

0.5.2

Breaking changes:

  • Chewy::Type::Base removed in favour of using Chewy::Type as a base class for all types

Changes

  • Chewy.massacre aliased to Chewy.delete_all method deletes all the indexes with current prefix

Bugs Fixed:

  • Advanced type classes resolving (@inbeom)
  • import ignores nil

0.5.1

Changes:

  • chewy.yml Rails generator (@jirikolarik)
  • Parent-child mappings feature support (@inbeom)
  • Chewy::Index.total_count and Chewy::Type::Base.total_count
  • Chewy::Type::Base.reset method. Deletes all the type documents and performs import (@jondavidford)
  • Added Chewy::Query#delete_all scope method using delete by query ES feature (@jondavidford)
  • Rspec 3 update_index matcher support (@jimmybaker)
  • Implemented function scoring (@averell23)

Bugs Fixed:

0.5.0

Breaking changes:

Changes:

  • suggest query options support (@rschellhorn).
  • Added hash data support. How it is possible to pass hashes to import.
  • rake chewy:reset and rake chewy:update paramless acts as rake chewy:reset:all and rake chewy:update:all respectively
  • Added delete_from_index? API method for custom deleted objects marking.
  • Added post_filter API, working the same way as filters.
  • Added chainable strategy query method.
  • Aliasing is performed in index create request for ElasticSearch >= 1.1.
  • preload scope method loads ORM/ODM objects in background.
  • load method :only and :except options to specify load types.
  • highlight and rescore query options support.
  • config/chewy.yml ERB support.

Bugs Fixed:

  • Fixed missing and exists filters DSL constructors.
  • Reworked index data composing.
  • Support for Kaminari new PaginatableArray behavior (@leemhenson)
  • Correct waiting for status. After index creation, bulk import, and deletion.
  • #23: Fix "wrong constant name" with namespace models

0.4.0

  • Changed update_index matcher behavior. Now it compare array attributes position-independently.
  • Search aggregations API support (@arion).
  • Chewy::Query#facets called without params performs the request and returns facets.
  • Added Type.template DSL method for root objects dynamic templates definition. See mapping.rb for more details.
  • ActiveRecord adapter custom primary_key support (@matthee).
  • Urgent update now clears association cache in ActiveRecord to ensure latest changes are imported.
  • import now creates index before performing.
  • Chewy.configuration[:wait_for_status] option. Can be set to red, yellow or green. If set - chewy will wait for cluster status before creating, deleting index and import. Useful for specs.

0.3.0

0.2.3

  • .import! indexes method, raises import errors.
  • .import! types method, raises import errors. Useful for specs.

0.2.2

  • Support for none scope (@undr).

  • Auto-resolved analyzers and analyzers repository (@webgago):

      # Setting up analyzers repository:
  Chewy.analyzer :title_analyzer, type: 'custom', filter: %w(lowercase icu_folding title_nysiis)
  Chewy.filter :title_nysiis, type: 'phonetic', encoder: 'nysiis', replace: false

  # Using analyzers from repository in index classes
  class ProductsIndex < Chewy::Index
    settings analysis: {analyzer: ['title_analyzer', {one_more_analyzer: {type: 'custom', tokenizer: 'lowercase'}}]}
  end

    title_analyzer here will be automatically resolved and passed to index mapping

0.2.0

  • Reworked import error handling. Now all the import errors from ElasticSearch are handled properly, also import method returns true of false depending on the import process success.

  • Chewy::Index.import now takes types hash as argument within options hash:

    PlacesIndex.import city: City.enabled, country: Country.enabled, refresh: false

  • Old indexes cleanup after reset.

  • Added index prefixes.

  • define_type now takes options for adapter.

  • chewy:reset and chewy:reset:all rake tasks are now trying to reset index with zero downtime if it is possible.

  • Added chewy:update:all rake task.

  • Methods .create, .create!, .delete, .delete, reset! are now supports index name suffix passing as the first argument. See actions.rb for more details.

  • Method reset renamed to reset!.

  • Added common loading scope for AR adapter. Also removed scope proc argument, now it executes just in main load scope context. CitiesIndex.all.load(scope: {city: City.include(:country)}) CitiesIndex.all.load(scope: {city: -> { include(:country) }}) CitiesIndex.all.load(scope: ->{ include(:country) })

0.1.0

  • Added filters simplified DSL. See filters.rb for more details.
  • Queries and filters join system reworked. See query.rb for more details.
  • Added query merge method
  • update_index matcher now wraps expected block in Chewy.atomic by default. This behaviour can be prevented with atomic: false option passing ruby expect { user.save! }.to update_index('users#user', atomic: false)
  • Renamed Chewy.observing_enabled to Chewy.urgent_update with false as default
  • update_elasticsearch renamed to update_index, added update_index :urgent option
  • Added import ActiveSupport::Notifications instrumentation ActiveSupport::Notifications.subscribe('import_objects.chewy') { |*args| }
  • Added types! and only! query chain methods, which purges previously chained types and fields
  • types chain method now uses types filter
  • Added types query chain method
  • Changed types access API: ruby UsersIndex::User # => UsersIndex::User UsersIndex::types_hash['user'] # => UsersIndex::User UsersIndex.user # => UsersIndex::User UsersIndex.types # => [UsersIndex::User] UsersIndex.type_names # => ['user']

  • update_elasticsearch method name as the second argument

      update_elasticsearch('users#user', :self)
  update_elasticsearch('users#user', :users)

  • Changed index handle methods, removed index_ prefix. I.e. was UsersIndex.index_create, became UsersIndex.create

  • Ability to pass value proc for source object context if arity == 0 field :full_name, value: ->{ first_name + last_name } instead of field :full_name, value: ->(u){ u.first_name + u.last_name }

  • Added .only chain to update_index matcher

  • Added ability to pass ActiveRecord::Relation as a scope for load CitiesIndex.all.load(scope: {city: City.include(:country)})

  • Added method all to index for query DSL consistency

  • Implemented isolated adapters to simplify adding new ORMs

  • Query DLS chainable methods delegated to index class (no longer need to call MyIndex.search.query, just MyIndex.query)

0.0.1

  • Query DSL
  • Basic index handling
  • Initial version