Releases: Namoshek/laravel-scout-database
Release candiate: Index Schema Overhaul, Fixes for Pagination and Improved Integration Test Coverage
This release comes with a lot of internal changes, which are reflected by a new database schema. The new schema may be adopted by applying the new database migration, which will also transfer all existing data in the search index. Changes to the database schema were made in an effort to reduce transaction deadlocks and the overall indexing and search performance.
A simple upgrade guide can be found in the README.
The full list of changes:
- Improve query performance through single-table index (#21)
- Support PHP 8 (#22)
- Improve query performance by replacing join (#23)
- Test on different DBMS using GitHub Actions (#24)
- Add tests for the DatabaseSeeker and fix small issues (#25)
- Cleanup test pipeline (#26)
- Add more tests for the search part of the package (#27)
- Prepare release candiate (#28)
Add support for Laravel 8
This release simply adds support for Laravel 8. No code changes were required.
Fix for failed jobs due to transaction deadlocks during indexing
With this release, a fix is being made available which is supposed to circumvent failing jobs due to transaction deadlocks during the indexing process. There is no full fix available for the issue at the moment, but granting multiple attempts to transactions can significantly reduce the amount of failed indexing jobs. More details can be found in #19.
Also noteworthy is that since the last release, a new automated test workflow has been added which runs for pull requests. The details can be found in #18. Although this has no impact on the actual code of the project, it seemed like a good opportunity to bump the minor version as well.
Performance Improvements and Tests for the DatabaseIndexer
With this release, which should be fully backwards compatible, some performance improvements have been added regarding the DatabaseIndexer (#6). It does now perform less queries when removing entries from the index (which also happens as part of an index update). There is also the option to disable cleaning of the words table as part of index updates now. The new CleanWordsTable command may be used instead, for example by scheduling it in app/Console/Kernel.php. For more details, see the README.
Furthermore, tests for the DatabaseIndexer have finally been added to ensure no negative side-effects occur (#7).
Fix potential precision issue with division in DatabaseSeeker
This release fixes a potential scoring issue which may be encountered using certain database engines due to the way they convert (or don't convert) data types to ones with higher precision when performing mathematical calculations, e.g. int to float (#5).
Fix potential division-by-zero in DatabaseSeeker
This release fixes a non-issue which may be encountered by database engines due to the way they optimize and validate queries (#4).
Fixes an Issue with the Data Type of passed Arguments #2
Data returned by the Laravel query system is converted to int when being passed to internal methods (25cd415).
Fixes an Issue with the Data Type of passed Arguments
Data returned by the Laravel query system is converted to int when being passed to internal methods (567bec0).
New NullStemmer, One Word List Entry per Document Type and Documentation Improvements
With this release, the documentation has been extended with a How does it work? section (1da2e80) and some limitations of the solution (5cffe74) were added as well.
A new NullStemmer has been added which can be used to disable the stemming process (9c16753). Be aware that this compromises the search functionality and will not allow to search for similar but not equal words. Also changing the stemmer requires to run indexing again.
Finally, the words table does now contain one entry per term and document_type to avoid search pollution by other document types (e2a51d1). Before, it was possible that when searching for documents of type A the inverse document frequency score has been low due to usages of the same word in documents of type B. This was especially problematic when documents of type B contained the searched word very frequently while documents of type A did not.
Upgrading from v0.2.0 to v0.3.0
As the project is still in some sort of beta phase, no semantic versioning is used. Therefore this update may be considered a major update in the semantic versioning scheme.
To upgrade, you'll have to add a new document_type column to the words table (configured table prefix omitted for clarity). Also the unique index on this table needs to be changed from the single term column to the columns ['document_type', 'term']. The easiest way to perform this task is to rollback the old migrations, publish the new ones and run them. However, re-indexing the search is required after performing the database upgrade.