What's New in Oracle Text?

This chapter describes new features of Oracle Text and provides pointers to additional information.

Oracle Database 10g R1 New Features

The following features are new for this release:

Security Improvements

In previous versions of Oracle Text, CTXSYS had DBA privileges. To tighten security and protect the database in the case of unauthorized access, CTXSYS now has only CONNECT and RESOURCE roles, and only limited, necessary direct grants on some system views and packages. Some applications using Oracle Text may therefore require minor changes in order to work properly with this security change.

Classification and Clustering

The following features are new for classification and clustering:

• Supervised Training and Document Classification

  The CTX_CLS.TRAIN procedure has been enhanced to support an additional classifier type called Support Vector Machine method for the supervised training of documents. The SVM method of training can produce better rules for classification than the query-based method.

  
  

  See Also: TRAIN in Chapter 6, "CTX_CLS Package" and the Oracle Text Application Developer's Guide

• Document Clustering

  The new CTX_CLS.CLUSTERING procedure enables you to generate document clusters. A cluster is a group of documents similar to each other in content.

  
  

  See Also: CLUSTERING in Chapter 6, "CTX_CLS Package"and the Oracle Text Application Developer's Guide

Indexing

The following features are new for indexing.

• Automatic and ON COMMIT Synchronization for CONTEXT index

  You can set the CONTEXT index to synchronize automatically either at intervals you specify or at commit time.

  
  

  See Also: Syntax for CONTEXT Indextype in Chapter 1, "Oracle Text SQL Statements and Operators".

• Transactional CONTEXT Indexes

  The new TRANSACTIONAL parameter to CREATE INDEX and ALTER INDEX enables changes to a base table to be immediately queryable.

  
  

  See Also: TRANSACTIONAL in Oracle Text SQL Statements and Operators

• Automatic Multi-Language Indexing

  The new WORLD_LEXER lexer type includes automatic language detection in documents, enabling you to index multilingual documents without having to include a language column in a base table.

  
  

  See Also: WORLD_LEXER in Chapter 2, " Oracle Text Indexing Elements"

• Mail Filtering

  Oracle Text can filter and index RFC-822 email messages. To do so, you use the new MAIL_FILTER filter preference.

  
  

  See Also: MAIL_FILTER in Chapter 2, " Oracle Text Indexing Elements"

• Fast Filtering of Binary Documents

  New attributes for the INSO_FILTER and MAIL_FILTER filter preferences offer the option of significantly improving performance when filtering binary documents. This fast filtering preserves only a limited amount of document formatting.

  
  

  See Also: INSO_FILTER and MAIL_FILTER in Chapter 2, " Oracle Text Indexing Elements"

• Support for creating local partitioned CONTEXT indexes in parallel

  You can now create local partitioned CONTEXT indexes in parallel with CREATE INDEX.

  
  

  See Also: CREATE INDEX in Chapter 1, "Oracle Text SQL Statements and Operators"

• MDATA section for adding metadata to documents

  You can now add an MDATA section to a section group. MDATA sections define metadata that enables you to perform mixed CONTAINS queries faster.

  
  

  See Also: ADD_MDATA and ADD_MDATA_SECTION in Chapter 7, "CTX_DDL Package"; MDATA in Chapter 3, "Oracle Text CONTAINS Query Operators"; the section searching chapter in the Oracle Text Application Developer's Guide

• ALTER TABLE enhanced support for partitioned tables

  ALTER TABLE supports the UPDATE GLOBAL INDEXES clause for partitioned tables.

  
  

  See Also: ALTER TABLE: Supported Partitioning Statements in Chapter 1, "Oracle Text SQL Statements and Operators"

• Binary Filtering for MULTI_COLUMN_DATASTORE

  The MULTI_COLUMN_DATASTORE now enables you to filter binary columns into text for concatenation with other columns during indexing. This datastore has also been enhanced to switch its XML-like auto-tagging on and off.

  
  

  See Also: MULTI_COLUMN_DATASTORE in Chapter 2, " Oracle Text Indexing Elements"

• New XML Output Option for Index Reports

  Several procedures and functions in the CTX_REPORT package now include a report_format parameter that enables you to obtain index report output either as plain text or XML.

  
  

  See Also: Chapter 11, "CTX_REPORT"

• Replacing Index Metadata

  You can replace index metadata (preference attributes) without having to rebuild the index. You do this using the new METADATA keyword with ALTER INDEX.

  
  

  See Also: ALTER INDEX REBUILD Syntax in Chapter 1, "Oracle Text SQL Statements and Operators"

• New Columns for Oracle Text Views

  Three Oracle Text views, CTX_OBJECT_ATTRIBUTES, CTX_INDEX_PARTITIONS, and CTX_USER_INDEX_PARTITIONS, have new columns.

  
  

  See Also: Appendix G, "Oracle Text Views"

• New Options for Index Optimization

  CTX_DDL.OPTIMIZE_INDEX has two new optlevels. TOKEN_TYPE optimizes on demand all tokens in the index matching the input token type. This is intended to help users keep critical field sections or MDATA sections optimal. REBUILD enables CTX_DDL.OPTIMIZE_INDEX to rebuild an index entirely.

  
  

  See Also: OPTIMIZE_INDEX in Chapter 7, "CTX_DDL Package"

• Log tokens During Index Optimization

  The CTX_OUTPUT.EVENT_OPT_PRINT_TOKEN event, which prints each token as it is being optimized, can be used with CTX_OUTPUT.ADD_EVENT.

  
  

  See Also: ADD_EVENT in Chapter 9, "CTX_OUTPUT Package"

• Tracing

  Oracle Text includes a tracing facility that enables you to identify bottlenecks in indexing and querying.

  
  

  See Also: ADD_TRACE in Chapter 9, "CTX_OUTPUT Package" and the Oracle Text Application Developer's Guide

• New German Spelling

  Oracle Text now can index German words under both traditional and reformed spelling.

  
  

  See Also: New German Spelling in Chapter 15, "Oracle Text Alternative Spelling"

Language Features

The following are new language features:

• Japanese Language Enhancements

  Oracle Text supports stem queries in Japanese with the stem $ operator.

  
  

  See Also: BASIC_WORDLIST in Chapter 2, " Oracle Text Indexing Elements" stem ($) operator in Chapter 3, "Oracle Text CONTAINS Query Operators"

• Customization of Japanese and Chinese Lexicons

  A new command, ctxlc, enables you to either modify the existing system Japanese and Chinese dictionaries (lexicons) or create new dictionaries from the merging of the system dictionaries with user-provided word lists. ctxlc also outputs the contents of dictionaries as word files.

  
  

  See Also: Lexical Compiler (ctxlc) in Chapter 14, "Oracle Text Executables"

• New character sets for the Chinese VGRAM lexer

  The Chinese VGRAM lexer now supports the AL32UTF8 and ZHS32GB18030 character sets.

  
  

  See Also: CHINESE_VGRAM_LEXER in Chapter 2, " Oracle Text Indexing Elements"

Querying

• Query Template Enhancements

  Query templating has been enhanced to provide the following features:

  • progressive relaxation of queries, which enables you to progressively execute less restrictive versions of a single query

  • query rewriting, which enables you to programatically rewrite any single query into different versions to increase recall

  • query language specification

  • alternative scoring algorithms

    
    

    See Also: CONTAINS in Chapter 1, "Oracle Text SQL Statements and Operators" The Querying chapter in the Oracle Text Application Developer's Guide

• Query Log Analysis

  Oracle Text now offers the capability to create a log of queries and to issue reports on its contents, indicating, for example, the most or least frequent successful queries.

  
  

  See Also: QUERY_LOG_SUMMARY in Chapter 11, "CTX_REPORT" START_QUERY_LOG and END_QUERY_LOG in Chapter 9, "CTX_OUTPUT Package"

• XML DB Enhancements

  Oracle Text has the following XML DB enhancements:

  • Better performance of existsNode()/CTXXPATH queries, with new support for attribute existence searching, and positional predicates.

  • Support for positional predicate testing with INPATH and HASPATH operators

    
    

    See Also: Syntax for CTXXPATH Indextype in Chapter 1, "Oracle Text SQL Statements and Operators" Oracle XML DB Developer's Guide

• Overriding of Base-letter Transformations

  A new BASIC_LEXER attribute, OVERRIDE_BASE_LETTER, prevents unexpected results when base-letter transformations are combined with alternate spelling.

  
  

  See Also: Overview of Alternative Spelling Features in Chapter 15, "Oracle Text Alternative Spelling"

Document Services

• Highlighting with INPATH and HASPATH

  Oracle Text supports highlighting with INPATH and HASPATH operators.

  
  

  See Also: Chapter 8, " CTX_DOC Package"

• CTX_DOC Enhancements for Policy-Based Document Services

  With the new CTX_DOC.POLICY_* procedures, you can perform document highlighting and filtering without requiring a table or a context index.

  
  

  See Also: Chapter 8, " CTX_DOC Package"