Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FTS extensions #1649

Merged
merged 29 commits into from
Apr 13, 2019
Merged

FTS extensions #1649

merged 29 commits into from
Apr 13, 2019

Conversation

MaceWindu
Copy link
Contributor

@MaceWindu MaceWindu commented Mar 16, 2019

fix #1561

  • [SQL Server] FREETEXTTABLE function support
  • [SQL Server] CONTAINSTABLE function support
  • [SQL Server] FREETEXT predicate support
  • [SQL Server] CONTAINS predicate support
  • [SQL Server] CONTAINS(PROPERTY()) predicate support
  • [SQLite] FTS extensions
  • [MySql] FTS extensions
  • Move FTS extensions to Sql.Ext.[dbname] extension points
  • (maybe) [PostgreSQL] FTS extensions postponed till user request

Summary:

  • [SQL Server] add new SQL Server extensions for FREETEXTTABLE/CONTAINSTABLE functions and FREETEXT/CONTAINSTEXT predicates with all kinds of parameters support
  • [SQL Server] obsolete old FreeTextTableExpressionAttribute extension attribute and Sql.FreeText() extension
  • [SQL Server] set default value of T4 GenerateSqlServerFreeText option to false to avoid obsoleted FreeTextTableExpressionAttribute-related FTS code generation
  • fix 'l => l.Tags' cannot be converted to SQL. when Join with FreeTextTable  #386 (by migrating to new extensions, obsoleted FreeTextTableExpressionAttribute is still broken and we don't plan to fix it)

@MaceWindu
Copy link
Contributor Author

@sdanyliv , @it - guys, we still need to decide new extensions naming schema

@MaceWindu
Copy link
Contributor Author

MaceWindu commented Mar 24, 2019

List of new extesions (for naming schema review).

MySQL:

// MATCH AGAINST as predicate
Sql.Ext.MySql().Match(search, params[] columns)
Sql.Ext.MySql().Match(modifier, search, params[] columns)
// MATCH AGAINST as relevance
Sql.Ext.MySql().MatchRelevance(search, params[] columns)
Sql.Ext.MySql().MatchRelevance(modifier, search, params[] columns)

@MaceWindu
Copy link
Contributor Author

SQLite:

// match(table) and match(column)
Sql.Ext.SQLite().Match(entityOrColumn, search);
Sql.Ext.SQLite().MatchTable(table, search);

// hidden columns
Sql.Ext.SQLite().RowId(table);
Sql.Ext.SQLite().Rank(table);

FTS3 and FTS5 functions:

// FTS3
Sql.Ext.SQLite().FTS3Offsets(table);
Sql.Ext.SQLite().FTS3MatchInfo(table);
Sql.Ext.SQLite().FTS3MatchInfo(table, format);
Sql.Ext.SQLite().FTS3Snippet(table);
Sql.Ext.SQLite().FTS3Snippet(table, startMatch);
Sql.Ext.SQLite().FTS3Snippet(table, startMatch, endMatch);
Sql.Ext.SQLite().FTS3Snippet(table, startMatch, endMatch, ellipses);
Sql.Ext.SQLite().FTS3Snippet(table, startMatch, endMatch, ellipses, columnIndex);
Sql.Ext.SQLite().FTS3Snippet(table, startMatch, endMatch, ellipses, columnIndex, tokensNumber);

// FTS5
Sql.Ext.SQLite().FTS5bm25(table);
Sql.Ext.SQLite().FTS5bm25(table, params[] weights);
Sql.Ext.SQLite().FTS5Highlight(table, columnIndex, startMatch, endMatch);
Sql.Ext.SQLite().FTS5Snippet(table, columnIndex, startMatch, endMatch, ellipses, tokensNumber);

FTS3 and FTS5 commands (DataConnection extensions):

// FTS3
dc.FTS3Optimize(table);
dc.FTS3Rebuild(table);
dc.FTS3IntegrityCheck(table);
dc.FTS3Merge(table, blocks, segments);
dc.FTS3AutoMerge(table, segments);

// FTS5
dc.FTS5AutoMerge(table, value);
dc.FTS5CrisisMerge(table, value);
dc.FTS5Delete(table, rowid, record);
dc.FTS5DeleteAll(table);
dc.FTS5IntegrityCheck(table);
dc.FTS5Merge(table, value);
dc.FTS5Optimize(table);
dc.FTS5Pgsz(table, value);
dc.FTS5Rank(table, function);
dc.FTS5Rebuild(table);
dc.FTS5UserMerge(table, value);

@MaceWindu
Copy link
Contributor Author

SQL Server:

// FREETEXTTABLE(table, *, ...)
Sql.Ext.SqlServer().FreeTextTable(table, search);
Sql.Ext.SqlServer().FreeTextTable(table, search, top);
Sql.Ext.SqlServer().FreeTextTableWithLanguage(table, search, top, language_name);
Sql.Ext.SqlServer().FreeTextTableWithLanguage(table, search, top, language_name, top);
Sql.Ext.SqlServer().FreeTextTableWithLanguage(table, search, top, language_code);
Sql.Ext.SqlServer().FreeTextTableWithLanguage(table, search, top, language_code, top);

// FREETEXTTABLE(table, (columns), ...)
Sql.Ext.SqlServer().FreeTextTable(table, columns_expr, search);
Sql.Ext.SqlServer().FreeTextTable(table, columns_expr, search, top);
Sql.Ext.SqlServer().FreeTextTableWithLanguage(table, columns_expr, search, top, language_name);
Sql.Ext.SqlServer().FreeTextTableWithLanguage(table,, columns_expr search, top, language_name, top);
Sql.Ext.SqlServer().FreeTextTableWithLanguage(table, columns_expr, search, top, language_code);
Sql.Ext.SqlServer().FreeTextTableWithLanguage(table, columns_expr, search, top, language_code, top);

// CONTAINSTABLE(table, *, ...)
Sql.Ext.SqlServer().ContainsTable(table, search);
Sql.Ext.SqlServer().ContainsTable(table, search, top);
Sql.Ext.SqlServer().ContainsTableWithLanguage(table, search, top, language_name);
Sql.Ext.SqlServer().ContainsTableWithLanguage(table, search, top, language_name, top);
Sql.Ext.SqlServer().ContainsTableWithLanguage(table, search, top, language_code);
Sql.Ext.SqlServer().ContainsTableWithLanguage(table, search, top, language_code, top);

// CONTAINSTABLE(table, (columns), ...)
Sql.Ext.SqlServer().ContainsTable(table, columns_expr, search);
Sql.Ext.SqlServer().ContainsTable(table, columns_expr, search, top);
Sql.Ext.SqlServer().ContainsTableWithLanguage(table, columns_expr, search, top, language_name);
Sql.Ext.SqlServer().ContainsTableWithLanguage(table,, columns_expr search, top, language_name, top);
Sql.Ext.SqlServer().ContainsTableWithLanguage(table, columns_expr, search, top, language_code);
Sql.Ext.SqlServer().ContainsTableWithLanguage(table, columns_expr, search, top, language_code, top);

// FREETEXT(table.*, ..)
Sql.Ext.SqlServer().FreeText(table, search);
Sql.Ext.SqlServer().FreeTextWithLanguage(table, search, language_name);
Sql.Ext.SqlServer().FreeTextWithLanguage(table, search, language_code);

// FREETEXT((columns), ..)
Sql.Ext.SqlServer().FreeText(table, search, param[] columns);
Sql.Ext.SqlServer().FreeTextWithLanguage(table, search, language_name, param[] columns);
Sql.Ext.SqlServer().FreeTextWithLanguage(table, search, language_code, param[] columns);

// CONTAINS(table.*, ..)
Sql.Ext.SqlServer().Contains(table, search);
Sql.Ext.SqlServer().ContainsWithLanguage(table, search, language_name);
Sql.Ext.SqlServer().ContainsWithLanguage(table, search, language_code);

// CONTAINS((columns), ..)
Sql.Ext.SqlServer().Contains(table, search, param[] columns);
Sql.Ext.SqlServer().ContainsWithLanguage(table, search, language_name, param[] columns);
Sql.Ext.SqlServer().ContainsWithLanguage(table, search, language_code, param[] columns);

// CONTAINS(PROPERTY())
Sql.Ext.SqlServer().ContainsProperty(column, property, search);
Sql.Ext.SqlServer().ContainsPropertyWithLanguage(column, property, search, language_name);
Sql.Ext.SqlServer().ContainsPropertyWithLanguage(column, property, search, language_code);

@MaceWindu MaceWindu changed the title [WIP] FTS extensions FTS extensions Apr 6, 2019
@MaceWindu
Copy link
Contributor Author

This is ready for review.
I decided to not add PostgreSQL FTS extensions now as it is another big chunk of work nobody requested yet. Could be added later on user request

/// <param name="columns">Full-text columns that should be queried.</param>
/// <returns>Returns <c>true</c> if full-text search found matching records.</returns>
[Sql.Extension("MATCH({columns, ', '}) AGAINST ({search})", IsPredicate = true, ServerSideOnly = true)]
public static bool Match(this IMySqlExtensions ext, [ExprParameter("search")] string search, [ExprParameter("columns")] params object[] columns)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no one place. But name of parameter is not required - original parameter name will be used

/// </summary>
public partial class TestDataDB : LinqToDB.Data.DataConnection
public partial class TestData2014DB : LinqToDB.Data.DataConnection
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we have to set name of DataConnection class explicitly.

Copy link
Member

@sdanyliv sdanyliv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work!

@MaceWindu MaceWindu merged commit bb54f89 into master Apr 13, 2019
@MaceWindu MaceWindu deleted the fts_extensions_v2 branch April 13, 2019 11:10
@MaceWindu
Copy link
Contributor Author

Whoops, forgot to address review comments, will do in master

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
4 participants