diff options
| author | server <brownptcdash@gmail.com> | 2019-12-10 18:12:37 -0500 |
|---|---|---|
| committer | server <brownptcdash@gmail.com> | 2019-12-10 18:12:37 -0500 |
| commit | 7478e610d99d1f2fb383ecbfa0b70d72eae27f81 (patch) | |
| tree | 59486701838067351f00580315c05690f36a39ce /solr-8.3.1/CHANGES.txt | |
| parent | 096718ee546afbd7568bf8ec9e23dca0556d814d (diff) | |
solr changes
Diffstat (limited to 'solr-8.3.1/CHANGES.txt')
| -rw-r--r-- | solr-8.3.1/CHANGES.txt | 19508 |
1 files changed, 19508 insertions, 0 deletions
diff --git a/solr-8.3.1/CHANGES.txt b/solr-8.3.1/CHANGES.txt new file mode 100644 index 000000000..219888b49 --- /dev/null +++ b/solr-8.3.1/CHANGES.txt @@ -0,0 +1,19508 @@ + Apache Solr Release Notes + +Introduction +------------ +Apache Solr is an open source enterprise search server based on the Apache Lucene Java +search library, with XML/HTTP and JSON APIs, hit highlighting, faceted search, +caching, replication, and a web administration interface. + +See http://lucene.apache.org/solr for more information. + + +Getting Started +--------------- +You need a Java 1.8 VM or later installed. +In this release, there is an example Solr server including a bundled +servlet container in the directory named "example". +See the Solr tutorial at https://lucene.apache.org/solr/guide/solr-tutorial.html + +================== 8.3.1 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.19.1 +Carrot2 3.16.0 +Velocity 2.0 and Velocity Tools 3.0 +Apache ZooKeeper 3.5.5 +Jetty 9.4.19.v20190610 + +Upgrade Notes +---------------------- + +* Velocity: The initialization parameters + "params.resource.loader.enabled" and "solr.resource.loader.enabled" have been removed and replaced by + Java system properties "velocity.resourceloader.params.enabled" and "velocity.resourceloader.solr.enabled" + respectively (both turned off by default). Please see SOLR-13971 for more details. + +Bug Fixes +---------------------- + +* SOLR-13963: JavaBinCodec has concurrent modification of CharArr resulting in corrupt internode updates (Colvin Cowie, noble) + +* SOLR-13941: Tests configure Jetty differently than when running via start.jar (janhoy, Uwe Schindler) + +* SOLR-13905: Make findRequestType in AuditEvent more robust (janhoy) + +* SOLR-13465: CoreContainer.auditloggerPlugin should be volatile (janhoy, hossman) + +* SOLR-13971: Velocity response writer's resource loading now possible only through startup parameters. + (Ishan Chattopadhyaya, David Smiley) + +================== 8.3.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.19.1 +Carrot2 3.16.0 +Velocity 2.0 and Velocity Tools 3.0 +Apache ZooKeeper 3.5.5 +Jetty 9.4.19.v20190610 + +Upgrade Notes +---------------------- + +* Users who have written test cases that extend SolrTestCaseJ4 may see NullPointerExceptions if + their tests directly reference both SolrTestCaseJ4.initCoreDataDir and SolrTestCaseJ4.deleteCore(). + This change in behavior is due to a bug fix in deleteCore() to ensure the dataDir is properly reset + in tests that call initCore()/deleteCore() multiple times in a given test (class). initCoreDataDir + is now deprecated, and users are encouraged to use SolrTestCaseJ4.initAndGetDataDir() in it's place. + See SOLR-13664 for more details. + +* For JWTAuthPlugin, the 'jwkUrl' configuration key is deprecated and may be removed later, please use 'jwksUrl' + instead. See SOLR-13734. + +New Features +---------------------- + +* SOLR-13375: Two dimensional routed aliases are now available for organizing collections based on the data values + of two fields. (Gus Heck) + +* SOLR-13399: SPLITSHARD implements a new splitByPrefix option that takes into account the actual document distribution + when using compositeIds. Document distribution is calculated using the "id_prefix" field (if it exists) containing + just the compositeId prefixes, or directly from the indexed "id" field otherwise. (yonik, Megan Carey) + +* SOLR-13622: Add cat() stream source to create tuples from lines in local files (Jason Gerlowski and Joel Bernstein) + +* SOLR-11866: QueryElevationComponent can have query rules configured with match="subset" wherein the words need only + match a subset of the query's words and in any order. (Bruno Roustant via David Smiley) + +* SOLR-13682: command line option to export documents to a file (noble) + +* SOLR-13257: Support deterministic replica routing preferences for better cache usage (Michael Gibney + via Christine Poerschke, Tomás Fernández Löbbe) + +* SOLR-13122: Ability to query aliases in Solr Admin UI (janhoy) + +* SOLR-13713: JWTAuthPlugin to support multiple JWKS endpoints (janhoy) + +* SOLR-13734: JWTAuthPlugin now supports multiple IdP issuers through configuring a new 'issuers' configuration key. + Access tokens issued and signed by any of the configured issuers will be validated (janhoy) + +* SOLR-13272: Add support for arbitrary ranges in JSON facet's Range facets. + (Apoorv Bhawsar, Munendra S N, Mikhail Khludnev, Ishan Chattopadhyaya, Jan Høydahl) + +* SOLR-13632: Support integral plots, cosine distance and string truncation with math expressions (Joel Bernstein) + +* SOLR-13667: Add upper, lower, trim and split Stream Evaluators (Joel Bernstein) + +* SOLR-13625: Add CsvStream, TsvStream Streaming Expressions and supporting Stream Evaluators (Joel bernstein) + +* SOLR-8241: Add CaffeineCache, an efficient implementation of SolrCache.(Ben Manes, Shawn Heisey, David Smiley, Andrzej Bialecki) + +Improvements +---------------------- + +* SOLR-12368: Support InPlace DV updates for a field that does not yet exist in any documents + (hossman, Simon Willnauer, Adrien Grand, Munendra S N) + +* SOLR-13558, SOLR-13693: Allow dynamic resizing of SolrCache-s. (ab) + +* SOLR-6305: Ability to set the replication factor for index files created by HDFSDirectoryFactory (Boris Pasko via Kevin Risden) + +* SOLR-13702: Some components register twice their metric names (janhoy) + +* SOLR-11601: Improved error message when geodist(llpsf) is used with arguments referring to a LatLonPointSpatialField. + (Amrit Sarkar) + +* SOLR-13542: Code cleanup - Avoid using stream filter count where possible (Koen De Groote via Tomás Fernández Löbbe) + +* SOLR-13720: BlockJoinParentQParser.getCachedFilter()made public for accessing from QParser plugins + (Stanislav Livotov via Mikhail Khludnev) + +* SOLR-13728: If a partial update (aka atomic update) is attempted on a document that has child docs, then ensure + the schema supports it (_root_ stored/docValues) by throwing an exception. (David Smiley) + +* SOLR-13742: Allow optional redaction of data saved by 'bin/solr autoscaling -save'. (ab) + +* SOLR-13739: Optimized large managed schema modifications; Internal O(n^2) problem. (Thomas Wöckinger via David Smiley) + +* SOLR-9658: Max idle time support for SolrCache implementations. (hoss, ab) + +* SOLR-13763: Improve the tracking of "freedisk" in autoscaling simulations. (ab) + +* SOLR-13773: Add Prometheus Exporter GC and Heap options. (Houston Putman via Anshum Gupta, David Smiley) + +* SOLR-13638: Add debug, trace logging to RuleBasedAuthorizationPlugin (Jason Gerlowski) + +* SOLR-13784: EmbeddedSolrServer's defaultCore constructor argument is now optional (David Smiley) + +* LUCENE-8984: MoreLikeThis MLT is biased for uncommon fields (Andy Hind via Anshum Gupta) + +* SOLR-13798: SSL: Adding Enabling/Disabling client's hostname verification config (Cao Manh Dat) + +* SOLR-13771: Add -v and -m to ulimit section of reference guide and bin/solr checks (Erick Erickson) + +* SOLR-13795: Managed schema operations should do a core reload in Solr standalone mode. + (Thomas Wöckinger via David Smiley) + +* SOLR-13719: Introducing SolrClient.ping(collection) in SolrJ (Geza Nagy via Mikhail Khludnev) + +Bug Fixes +---------------------- + +* SOLR-13206: Fix AIOOBE when group.facet is specified with group.query and return proper error code. (Marek, Munendra S N) + +* SOLR-11556: Backup and restore command really supports picking one of a few repositories by + repository parameter (Timothy Potter via Mikhail Khludnev) + +* SOLR-13647: Default solr.in.sh contains incorrect default value (John Ryan) + +* SOLR-13660: Fixed AbstractFullDistribZkTestBase.waitForActiveReplicaCount() to ensure + replicas are active. (hossman) + +* SOLR-13664: Fixed SolrTestCaseJ4.deleteCore() to properly reset the dataDir used by initCore() + (hossman) + +* SOLR-13679: Default style of ExplainDocTransformer registered via solrconfig.xml should be same as default style + of ExplainDocTransformer registered in defaultFactories (Munendra S N) + +* SOLR-13672: Cloud -> Zk Status page now parses response from Zookeeper 3.5.5 correctly (Jörn Franke, janhoy, Shawn Heisey) + +* SOLR-13674: NodeAddedTrigger does not support configuration of replica type hint. A new replicaType property + has been added to NodeAddTrigger so that new replicas of the given type are added. The default value is `NRT`. + (Irena Shaigorodsky via shalin) + +* SOLR-13141: CDCR bootstrap does not replicate index to the replicas of target cluster. + (Krzysztof Watral, Amrit Sarkar, Tim, Tdspringsteen, shalin) + +* SOLR-13694: IndexSizeEstimator NullPointerException. (ab, hossman) + +* SOLR-13700: Fixed a race condition when initializing metrics for new security plugins on security.json change (hossman) + +* SOLR-13701: Fixed JWTAuthPlugin to update metrics prior to continuing w/other filters or returning error (hossman) + +* SOLR-6328: facet.missing=true should return missing count even when facet.limit=0 (hossman, Munendra S N) + +* SOLR-13704: Improve error message and change error code to 400 for client errors in ExpandComponent (Munendra S N) + +* SOLR-13699 - maxChars no longer working on CopyField with javabin (Chris Troullis via noble) + +* SOLR-13718: SPLITSHARD (async) with failures in underlying sub-operations can result in data loss (Ishan Chattopadhyaya) + +* SOLR-13717: Fixed distributed grouping when multiple 'fl' params are specified (hossman, Christine Poerschke) + +* SOLR-13240: Fixed UTILIZENODE action resulting in IllegalArgumentException. + (Hendrik Haddorp, Richard Goodman, Tim Owen, shalin, noble, Christine Poerschke) + +* SOLR-13238: BlobHandler generates non-padded md5 (Jeff Walraven via janhoy) + +* SOLR-13780: Fix ClassCastException in NestableJsonFacet (Tiago Martinho de Barros, Munendra S N) + +* SOLR-13725: Allow negative values for limit in TermsFacetMap (Richard Walker, Munendra S N) + +* SOLR-13022: Fix NPE when sorting by non-existent aggregate function in JSON Facet (hossman, Munendra S N) + +* SOLR-13727: Fixed V2Requests - HttpSolrClient replaced first instance of "/solr" with "/api" which + caused a change in host names starting with "solr". (Megan Carey via yonik) + +* SOLR-13180: Fix ClassCastException in Json Request API (Johannes Kloos, Jan Høydahl, Munendra S N) + +* SOLR-13417: Handle stats aggregation on date and string fields in SolrJ's JSON facet response processing + (Jason Gerlowski, Munendra S N) + +* SOLR-13712: JMX MBeans are not exposed because of race condition between creating platform mbean server and + registering mbeans. (shalin) + +* SOLR-13802: Managed schema manipulations were not persisting the optional luceneMatchVersion that can be set + on an Analyzer. (Thomas Wöckinger) + +* SOLR-13790: LRUStatsCache size explosion and ineffective caching. (ab) + +* SOLR-13539: Fix for class-cast issues during atomic-update 'removeregex' operations. This also incorporated some + tests Tim wrote as a part of SOLR-9505. (Tim Owen via Jason Gerlowski) + +* SOLR-13376: Multi-node race condition to create/remove nodeLost markers. (hoss, ab) + +* SOLR-13293: ConcurrentUpdateHttp2SolrClient always log AsynchronousCloseException exception error on indexing. + (Cao Manh Dat) + +* SOLR-13828: Improve ExecutePlanAction error handling. (ab) + +* SOLR-13760: Fix regression in support for Date math in TRA start date that was introduced by SOLR-13375 + +* SOLR-13829: RecursiveEvaluator casts Continuous numbers to Discrete Numbers, causing mismatch (Trey Grainger, Joel Bernstein) + +* SOLR-13815: Live shard split (where updates actively continue during the split) can lose updates due to cluster + state happening to change between checking if the current shard is active and later checking if there are any + sub-shard leaders to forward the update to. (yonik) + +* SOLR-13665: Added missing netty dependencies to solrJ and upgraded netty to v 4.1.29.Final (Jörn Franke, janhoy) + +* SOLR-13793: HttpSolrCall now maintains internal request count (_forwardedCount) for remote queries and limits them to + the number of replicas. This avoids making too many cascading calls to remote servers, which, if not restricted, can + bring down nodes containing the said collection (Kesharee Nandan Vishwakarma, Ishan Chattopadhyaya) + +* SOLR-13834: ZkController#getSolrCloudManager() created a new instance of ZkStateReader, thereby causing mismatch in the + visibility of the cluster state and, as a result, undesired race conditions (Clay Goddard via Ishan Chattopadhyaya) + +* SOLR-13835: HttpSolrCall produces incorrect extra AuditEvent on AuthorizationResponse.PROMPT (janhoy, hossman) + +* SOLR-13843: The MOVEREPLICA API ignores replica type and always adds 'nrt' replicas (Amrit Sarkar via shalin) + +* SOLR-13677: All Metrics Gauges should be unregistered by components that registered them. (noble, ab) + +* SOLR-13855: DistributedZkUpdateProcessor should have been propagating URP.finish() lifecycle like it used to before + 8.1 (a regression). Impacts integrity since Run URP's finish() propagates this to the updateLog to fsync. + (David Smiley) + +Other Changes +---------------------- + +* SOLR-13779: Use the safe fork of simple-xml for clustering contrib. (Dawid Weiss) + +* SOLR-13585: Factor out SearchGroupsResultTransformer.[de]serializeOneSearchGroup methods. (Christine Poerschke, Diego Ceccarelli) + +* SOLR-12870: Use StandardCharsets instead of String values (Peter Somogyi via Munendra S N) + +* SOLR-10377: Add `debug.explain.structured` to Admin UI. (David Smiley, Munendra S N) + +* SOLR-13629: Clean trailing whitespace from 'analytics' contrib (Neal Sidhwaney via Jason Gerlowski) + +* SOLR-13643: Add Getters/Setters in ResponseBuilder for analytics response handling (Neal Sidhwaney via Munendra S N) + +* SOLR-13680: Use try-with-resource to close the closeable resource (Furkan KAMACI, Munendra S N) + +* SOLR-13573: Add SolrRangeQuery getters for upper, lower bound (Brian Rhees via Jason Gerlowski) + +* SOLR-13658: Precommit fail Java "var" until 9x. Fail "var...<>" constructs entirely (Erick Erickson) + +* SOLR-13767: Upgrade jackson to 2.9.9 (janhoy) + +* SOLR-11492: Clean up /solr/cloud-dev scripts and provide a single well documented script (Gus Heck, Robert Bunch) + +* SOLR-13747: New TestSSLTestConfig.testFailIfUserRunsTestsWithJVMThatHasKnownSSLBugs() to give people running + tests more visibility if/when they use a known-buggy JVM causing most SSL tests to silently SKIP. (hossman) + +* SOLR-13791: Remove remaining Commons BeanUtils references. (Andras Salamon, Christine Poerschke) + +* SOLR-13812: Add javadocs, uneven rejection and basic test coverage for the SolrTestCaseJ4.params method. + (Diego Ceccarelli, Christine Poerschke, Munendra S N) + +* SOLR-13787: An annotation based system to write v2 APIs (noble) + +* SOLR-12786: Update Ref Guide build tool versions (Cassandra) + +================== 8.2.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.19.1 +Carrot2 3.16.0 +Velocity 2.0 and Velocity Tools 3.0 +Apache ZooKeeper 3.5.5 +Jetty 9.4.19.v20190610 + +Upgrade Notes +---------------------- + +* SOLR-8346: Upgrade ZooKeeper to 3.5.5. ZooKeeper 3.5.5 introduces more security features. Include in your zoo.cfg + file at minimum the following: + 4lw.commands.whitelist=mntr,conf,ruok + You can use + 4lw.commands.whitelist=* to enable all ZooKeeper "4 letter commands". + (Erick Erickson) + +* SOLR-13515: org.apache.solr.util.SolrPluginUtils.IdentityRegenerator has been removed, please use + the identical org.apache.solr.search.NoOpRegenerator instead. (David Smiley, Christine Poerschke) + +* SOLR-13541: Upgrade Jetty to 9.4.19.v20190610. (Erick Erickson, Cao Manh Dat) + +* SOLR-13420: Routed Aliases now use collection properties rather than core properties to identify collections that + belong to the alias by default. This should be invisible and fully backwards compatible from within solr, and + existing routed alias collections with core based properties will continue to work, but new collections created will + not add a property to core.properties anymoore so any external code that inspected core.properties will not find the + 'routedAliasName' key in new cores belonging to routed aliases. + +* SOLR-13419: Time Routed Aliases now have a '__TRA__' infix preceding the timestamp portion of the collection names. + collections with the old format will not be altered and will continue to work, but any new TRA's or new collections + for existing TRA's will use the new format. Solr will handle this invisibly, but any external code that attempted to + predict collection names in TRA's will probably need adjustment. + +* SOLR-13507: Support for "addr" parameter from the "/solr/admin/zookeeper" endpoint has now been removed and will no + longer be supported. + +New Features +---------------------- + +* SOLR-13320: add an update param failOnVersionConflicts=false to updates not fail when there is a version conflict (noble) + +* SOLR-13445: Preferred replicas on nodes with same system properties as the query master (Cao Manh Dat) + +* SOLR-13047: Add facet2D Streaming Expression (Nazerke Seidan, Joel Bernstein) + +* SOLR-13494: Add DeepRandomStream implementation (Joel Bernstein) + +* SOLR-13434: OpenTracing support for Solr (Cao Manh Dat) + +* SOLR-13512: Raw index data analysis tool (extension of COLSTATUS collection command). (ab) + +* SOLR-13550: Allow zplot to automatically create the x axis (Joel Bernstein) + +* SOLR-13552: Add recNum Stream Evaluator (Joel Bernstein) + +* SOLR-13560: Add isNull and notNull Stream Evaluators (Joel Bernstein) + +* SOLR-10291: Add matches Stream Evaluator to support regex matching (Joel Bernstein) + +* SOLR-13589: Allow zplot to visualize 2D clusters and convex hulls (Joel Bernstein) + +Improvements +---------------------- + +* SOLR-13049: Make contrib/ltr Feature.defaultValue configurable. (Stanislav Livotov, Christine Poerschke) + +* SOLR-13306: Add a request parameter to execute a streaming expression locally + +* SOLR-13440: Support saving/restoring autoscaling state for repeatable simulations. (ab) + +* SOLR-11558: It would be nice if the Graph section of the Cloud tab in the Admin UI could give some more + information about the replicas of a collection (Erick Erickson) + +* SOLR-13468: autoscaling/suggestions should be able to give suggestions from config sent as a payload (noble) + +* SOLR-12304: The MoreLikeThisComponent now supports the mlt.interestingTerms parameter. Previously this option was + unique to the MLT handler. (Alessandro Benedetti via David Smiley) + +* SOLR-13484: autoscaling/diagnostics API should be able to give diagnostics output from config sent as a payload (noble) + +* SOLR-13493: /autoscaling/suggestions to be able to filter by type (noble) + +* SOLR-13504: In autoscaling policies, use an explicit 'nodeset' attribute for filtering + nodes instead of using them directly at the toplevel (noble) + +* SOLR-13329: In autoscaling policies, use an explicit 'put : on-each-node' + to specify the the rules is applied on each node (noble) + +* SOLR-13347: Transaction log to natively support UUID types (Thomas Wöckinger via noble) + +* SOLR-13367: Highlighting: Range queries will now highlight in hl.method=unified mode. (David Smiley) + +* SOLR-13569: Admin UI visual indication of prod/test/dev environment (janhoy) + +* SOLR-12554: Expose IndexWriterConfig's ramPerThreadHardLimitMB through solrconfig.xml (Ishan Chattopadhyaya, Munendra S N) + +* SOLR-13602 SOLR-13588: Add a field type for Estonian language to default managed_schema, + document about Estonian language analysis in Solr Ref Guide (Tomoko Uchida) + +* SOLR-13003: Query Result Cache does not honor maxRamBytes parameter. (ab, Brian Ecker) + +Bug Fixes +---------------------- + +* SOLR-13474: Fix "Search is temporarily disabled" logic to be consistent for entire request (hossman) + +* SOLR-13469: Fix rejected requests during full disk full + IndexFetch to use 503 not 403 (hossman) + +* SOLR-12941: Fix IndexSizeTrigger to correctly work with "aboveBytes" and "splitMethod=link" parameters. (ab) + +* SOLR-13491: SolrZkClient's watch wrapper no longer allows zookeeper to hold the same watch object multiple times. + +* SOLR-13489: Stop the leader from trying to rejoin the election on session expiration and harden our zk reconnect code path. + (Mark Miller, Anshum Gupta) + +* SOLR-12249: Better error message when grouping on a tokenized (non SortableText) field in SolrCloud (Erick Erickson) + +* SOLR-13521: Fix input parameter handling in SchemaRequest.DynamicField and SchemaRequest.FieldTypes (Schema API) + (Tomás Fernández Löbbe) + +* SOLR-13496: Fix distributed grouping related NullPointerException in JSONWriter.writeSolrDocument (Christine Poerschke) + +* SOLR-12013: collections API CUSTERSTATUS command fails when configset missing (Erick Erickson) + +* SOLR-13509: NPE on omitHeader=true is fixed by sending omitHeader=false to shard searches (Munendra S N, Mikhail Khludnev) + +* SOLR-13333: unleashing terms.ttf from terms.list when distrib=false (Munendra S N via Mikhail Khludnev) + +* SOLR-13490: Fix CollectionStateWatcher/CollectionStatePredicate based APIs in ZkStateReader and + CloudSolrClient to be triggered on liveNode changes. Also add Predicate<DocCollection> equivalents + for callers that don't care about liveNodes. (hossman) + +* SOLR-13545: ContentStreamUpdateRequest refused to close file (Colvin Cowie, Mikhail Khludnev) + +* SOLR-12127: Updates containing 'set' operation with value null or empty list should be considered as Atomic Update + (Oliver Kuldmäe, Munendra S N, Ishan Chattopadhyaya) + +* SOLR-13187: Fix NPE when invalid query parser is specified and return 400 error code. + (Cesar Rodriguez, Marek, Charles Sanders, Munendra S N, Mikhail Khludnev) + +* SOLR-12979: Improve error message and change error code to 400 when collapse field is non-docValued and + non-uninvertible (hossman, Munendra S N) + +* SOLR-13574: Fix many test and test-framework classes to not fail on After/AfterClass cleanup if + assumptions fail in Before/BeforeClass setup (hossman) + +* SOLR-13280: Strengthen ScheduledTrigger's preferredOperation parameter validation. (Christine Poerschke) + +* SOLR-9409: Improve error message for unsupported field types and return 400 error code on + unsupported values in collapsing (hossman, Munendra S N) + +* SOLR-13404: Support group.query in multi-shard environment when group.main=true or group.format=simple (Munendra S N) + +* SOLR-13538: toNativeType () TrieDate & EnumField do not handle CharSequence properly (Munendra S N) + +* SOLR-13532: Fix http timeout and error logging bugs in RecoveryStrategy. (Suril Shah, hossman) + +* SOLR-13472: Forwarded requests should skip authorization on receiving nodes (adfel, Ishan Chattopadhyaya) + +* SOLR-13619: Kerberos plugin to pass original user principal to avoid 403 on nodes not hosting a collection + (adfel, Ishan Chattopadhyaya, noble) + +Other Changes +---------------------- + +* SOLR-13453: Adjust auth metrics asserts in tests caused by SOLR-13449 (janhoy) + +* SOLR-13437: noggit json parser is forked into solrj (noble) + +* SOLR-13467: Include the S2 Geometry lib to make it simpler to use prefixTree="s2" on a Geo3D spatial field. + Improved documentation on Geo3D too. (David Smiley) + +* SOLR-13371: Re-structure and clarify Security chapters in Ref Guide (janhoy) + +* SOLR-13235: Split Collections API Ref Guide page into several smaller child pages (Cassandra Targett) + +* SOLR-8754: Adding test cases for org.apache.solr.util.hll.NumberUtilTest (Benoit Vanalderweireldt via janhoy) + +* SOLR-13511: Add SearchHandler.newResponseBuilder method to facilitate custom plugins' maintenance + of per-request state in a custom ResponseBuilder. (Ramsey Haddad, Christine Poerschke) + +* SOLR-12988: SSLTestConfig has been changed to throw AssumptionViolatedException when tests/seeds + request SSL but the JVM appears to be an OpenJDK version known to have SSL bugs (hossman, Cao Manh Dat) + +* SOLR-13279: Clarify ScheduledTrigger's "every parameter missing" error response. (Christine Poerschke) + +* SOLR-13576: Factor out a TopGroupsShardResponseProcessor.fillResultIds method. (Christine Poerschke, Diego Ceccarelli) + +* SOLR-12364: Add test cases for edismax boost (David Smiley, Munendra S N) + +* SOLR-13603: Remove usage of GroupingSpecification's deprecated methods (Munendra S N) + +* SOLR-13507: Remove support for "addr" parameter from the "/solr/admin/zookeeper" endpoint. (janhoy, Anshum Gupta) + +* SOLR-13588: Document Estonian analyzer in Solr Ref Guide (Tomoko Uchida) + +* SOLR-13602: Add a field type for Estonian language to default managed_schema (Tomoko Uchida) + +================== 8.1.2 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.19.1 +Carrot2 3.16.0 +Velocity 2.0 and Velocity Tools 3.0 +Apache ZooKeeper 3.4.14 +Jetty 9.4.14.v20181114 + +Bug Fixes +---------------------- + +* SOLR-13510: Intermittent 401's for internode requests with basicauth enabled (Cao Manh Dat, Colvin Cowie) + +* SOLR-13523: In 8.1, Atomic Updates were broken (NPE) when the schema declared the new _nest_path_ field even if you + weren't using nested docs. In-place updates were not affected (worked). + Reminder: if you don't need nested docs then remove both _root_ and _nest_path_ ! (David Smiley) + +* SOLR-13566: REINDEXCOLLECTION does not work with (basic) authentication. (Colvin Cowie, ab) + +* SOLR-13159: Fix atomic update encoding issue for UUID, enum, bool, and binary fields (Thomas Wockinger via Jason Gerlowski) + +* SOLR-13583: Impossible to delete a collection with the same name as an existing alias. This fixes also a bug in + REINDEXCOLLECTION when used with removeSource=true which could lead to a data loss. (ab) + +Improvements +---------------------- + +* SOLR-13158: DataImportHandler: Added enable.dih.dataConfigParam system property to toggle whether the dataConfig param + is permitted. (David Smiley, janhoy, Tomás Fernández Löbbe) + +================== 8.1.1 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.19.1 +Carrot2 3.16.0 +Velocity 2.0 and Velocity Tools 3.0 +Apache ZooKeeper 3.4.14 +Jetty 9.4.14.v20181114 + +Bug Fixes +---------------------- +* SOLR-13475: Null Pointer Exception when querying collection through collection alias. (Jörn Franke, ab) + +================== 8.1.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.19.1 +Carrot2 3.16.0 +Velocity 2.0 and Velocity Tools 3.0 +Apache ZooKeeper 3.4.14 +Jetty 9.4.14.v20181114 + +Upgrade Notes +---------------------- + +* Solr's default behavior when dealing with 'maxBooleanClauses' has changed to reduce the risk of exponential + query expansion when dealing with pathological query strings. A default upper limit of 1024 clauses + (The same default prior to Solr 7.0) is now enforced at the node level, and can be overridden in solr.xml. + The identically named solrconfig.xml setting is still available for limiting the size of 'explicit' boolean + query strings, but this per-collection limit is still ristricted by the upper-bound of the global limit + in solr.xml. See SOLR-13336 for more details. + +* When requesting the status of an async request via REQUESTSTATUS collections API, the response will + include the list of internal async requests (if any) in the "success" or "failed" keys (in addition + to them being included outside those keys for backwards compatibility). See SOLR-12708 for more + details + +* SOLR-12891: MacroExpander will no longer will expand URL parameters inside of the 'expr' parameter (used by streaming + expressions) Additionally, users are advised to use the 'InjectionDefense' class when constructing streaming + expressions that include user supplied data to avoid risks similar to SQL injection. The legacy behavior of + expanding the 'expr' parameter can be reinstated with -DStreamingExpressionMacros=true passed to the JVM at startup + (Gus Heck). + +* SOLR-13335: Velocity and Velocity Tools were both upgraded as part of this release. Velocity upgraded from 1.7 to 2.0. + Please see https://velocity.apache.org/engine/2.0/upgrading.html about upgrading. Velocity Tools upgraded from 2.0 to + 3.0. For more details, please see https://velocity.apache.org/tools/3.0/upgrading.html for details about the upgrade. + +* SOLR-13407: Update requests sent to non-routed aliases that point to multiple collections are no longer accepted. Until + now Solr followed an obscure convention of updating only the first collection from the list, which usually was not what + the user intended. This change explicitly rejects such update requests. + +* SolrGangliaReporter has been removed from Solr because support for Ganglia has been removed from Dropwizard Metrics 4 + due to a transitive dependency on LGPL. + +* Custom TransientSolrCoreCache implementations no longer use the Observer/Observable pattern. To notify Solr that + a core has been aged out of the cache, call CoreContainer.queueCoreToClose(SolrCore). See SOLR-13400 for details. + +* SOLR-13394: The default GC has been changed from CMS to G1. To override this (in order to switch to CMS or any + other GC), use GC_TUNE section of bin/solr.in.sh or bin/solr.in.cmd. + +* SOLR-5970: Until now, the CREATE command of Collections API returned status 0 even in case of failure. Now, the status + code will be non-zero in case of failures (e.g. 4xx, 5xx). + +New Features +---------------------- +* SOLR-13131: Category Routed Aliases are now available for data driven assignment of documents to collections based on + values of a field. The Ref Guide now has a page dedicated to explaining the different types of aliases. (Gus Heck, + Moshe Bla) + +* SOLR-13171 : A true streaming parser for javabin payload/stream without creating any objects (noble) + +* SOLR-13261: Make SortableTextField work with export/streaming. NOTE: requires that the field have + useDocValuesAsStored=true (either explicit or as the default). + +* SOLR-12121: JWT Token authentication plugin with OpenID Connect implicit flow login through Admin UI (janhoy) + +* SOLR-12120: New AuditLoggerPlugin type allowing custom Audit logger plugins (janhoy) + +* SOLR-10436: Add hashRollup Streaming Expression (Joel Bernstein) + +* SOLR-13276: Adding Http2 equivalent classes of CloudSolrClient and HttpClusterStateProvider (Cao Manh Dat) + +* SOLR-13287: Allow zplot to visualize probability distributions in Apache Zeppelin (Joel Bernstein) + +* SOLR-13271: Read-only mode for SolrCloud collections (ab, shalin) + +* SOLR-13292: Provide extended per-segment status of a collection. (ab) + +* SOLR-11127: REINDEXCOLLECTION command for re-indexing of existing collections. This issue also adds + a back-compat check of the .system collection to notify users of potential compatibility issues after + upgrades or schema changes. (ab) + +* SOLR-13374: Add fetchSize parameter to the jdbc Streaming Expression (Joel Bernstein) + +* SOLR-12638: Partial/Atomic Updates for nested documents. This enables atomic updates for nested documents, without + the need to supply the whole nested hierarchy (which would be overwritten if absent). This is done by fetching the + whole document hierarchy, updating the specific doc in the path that is to be updated, removing the old document + hierarchy and indexing the new one with the atomic update merged into it. Also, [child] Doc Transformer now works + with RealTimeGet. (Moshe Bla, David Smiley) + +* SOLR-13262: Add collection RENAME command and support using aliases in most collection admin commands. (ab) + +* SOLR-13391: Add variance and standard deviation stream evaluators (Nazerke Seidan, Joel Bernstein) + +* SOLR-13427: Support simulating the execution of autoscaling suggestions. (ab) + +Bug Fixes +---------------------- + +* SOLR-12330: 500 error code on json.facet syntax errors (Munendra S N, Mikhail Khludnev) + +* SOLR-13229: Cleanup replicasMetTragicEvent after all types of exception (Tomás Fernández Löbbe) + +* SOLR-11876: In-place update fails when resolving from Tlog if schema has a required field (Justin Deoliveira, janhoy, + Ishan Chattopadhyaya) + +* SOLR-12708: Async collection actions should not hide internal failures (Mano Kovacs, Varun Thacker, Tomás Fernández Löbbe) + +* SOLR-11883: 500 code on functional query syntax errors and parameter dereferencing errors +(Munendra S N via Mikhail Khludnev) + +* SOLR-13285: Updates with enum fields and javabin cause ClassCastException (noble) + +* SOLR-13234: Prometheus Metric Exporter not threadsafe. This changes the prometheus exporter to collect metrics + from Solr on a fixed interval controlled by this tool and prevents concurrent collections. This change also improves + performance slightly by using the cluster state instead of sending multiple HTTP requests to each node to lookup + all the cores. + (Danyal Prout via shalin) + +* SOLR-9882: 500 error code on breaching timeAllowed by core and distributed (fsv) search, + old and json facets (Mikhail Khludnev) + +* SOLR-13295: Reproducible failure in TestDistributedGrouping (Erick Erickson) + +* SOLR-13254: Correct message that is logged in solrj's ConnectionManager when an exception + occurred while reconnecting to ZooKeeper. (hu xiaodong via Christine Poerschke) + +* SOLR-13284: NullPointerException with 500 http status on omitted or wrong wt param. + It's fixed by fallback to json (Munendra S N via Mikhail Khludnev) + +* SOLR-13244: Admin UI Nodes view fails and is empty when a node is temporarily down (janhoy) + +* SOLR-13253: IndexSchema.getResourceLoader was being used to load non-schema things, which can be a memory leak if + "shareSchema" and other circumstances occur. Furthermore it's reference to SolrConfig was removed. (David Smiley) + +* SOLR-7414: CSVResponseWriter & XLSXResponseWriter return empty field when fl alias is combined with '*' selector + (Michael Lawrence, Munendra S N, Ishan Chattopadhyaya) + +* SOLR-13351: Workaround for VELOCITY-908 (Kevin Risden) + +* SOLR-13349: High CPU usage in Solr due to Java 8 bug (Erick Erickson) + +* SOLR-13344: Admin UI inaccessible with RuleBasedAuthorizationPlugin (janhoy, Jason Gerlowski) + +* SOLR-13352: Remove risk of deadlock/threadleak when shutting down an Overseer(TriggerThread). (hossman) + +* SOLR-13362: Add 'includeIndexFieldFlags' support to SolrJ LukeRequest (Jason Gerlowski) + +* SOLR-13355: 'all' permission ignored by RuleBasedAuthorizationPlugin in most cases (Jason Gerlowski, janhoy) + +* SOLR-13331: Atomic Update 'remove' operations broken for certain field types in SolrJ (Thomas Wockinger via Jason Gerlowski) + +* SOLR-13388: Fix FileExchangeRateProvider to be a public class, as it appears in schema.xml (Uwe Schindler) + +* SOLR-12860: MetricsHistoryHandler now uses PKI Auth for metrics collection in background thread (janhoy, Lorenzo) + +* SOLR-13339: Prevent recovery, fetching index being kicked off after SolrCores already closed (Cao Manh Dat) + +* SOLR-13393: Fixed ZkClientClusterStateProvider to prevent risk of leaking ZkStateReader/threads when + processing concurrent requests during shutdown. This primarily affected tests, but may have also caused + odd errors/delays when restart/shutting down solr nodes. (hossman) + +* SOLR-13336: add maxBooleanClauses (default to 1024) setting to solr.xml, reverting previous effective + value of Integer.MAX_VALUE-1, to restrict risk of pathalogical query expansion. (hossman) + +* SOLR-13386: OverseerTaskQueue#remove should not throw an exception when no node exists after an exists + check and the Overseer work loop should not allow free spinning the loop when it hits a KeeperException. + (Mark Miller, Fernandez-Lobbe, Mike Drob) + +* SOLR-12371: Editing authorization config via REST API now works in standalone mode (janhoy) + +* SOLR-13408: Cannot start/stop DaemonStream repeatedly, other API improvements (Erick Erickson) + +* SOLR-13281: Fixed NPE in DocExpirationUpdateProcessorFactory (Munendra S N, Tomás Fernández Löbbe) + +* SOLR-13081: In-Place Update doesn't work with route.field (Dr Oleg Savrasov via Mikhail Khludnev) + +* SOLR-13343: Fix spacing issue in browser log (Marcus Eagan via Jason Gerlowski) + +* SOLR-12248, SOLR-4647: Grouping is broken on docValues-only fields (non-stored, non-indexed) + (Erick Ericsson, Adrien Grand, Munendra S N, Scott Stults, Ishan Chattopadhyaya) + +* SOLR-5970: Return correct status upon collection creation failure (Abraham Elmahrek, Ishan Chattopadhyaya, + Jason Gerlowski, Kesharee Nandan Vishwakarma) + +* SOLR-12291: prematurely reporting not yet finished async Collections API call as completed + when collection's replicas are collocated at least at one node (Varun Thacker, Mikhail Khludnev) + +* SOLR-13410: Designated overseer wasn't able to rejoin election queue upon restart (Ishan Chattopadhyaya, + Kesharee Nandan Vishwakarma) + +* SOLR-13318: Fix ClassCastException in SolrJ JsonFaceting classes (Munendra S N via Jason Gerlowski) + +* SOLR-13449: SolrClientNodeStateProvider always retries on requesting metrics from other nodes (Cao Manh Dat) + +Improvements +---------------------- + +* SOLR-12999: Index replication could delete segments before downloading segments from master if there is not enough + disk space (noble) + +* SOLR-12055 introduces async logging by default. There's a small window where log messages may be lost + in the event of some hard crash. Switch back to synchronous logging if this is unacceptable, see + see commeints in the log4j2 configuration files (log4j2.xml by default). + +* SOLR-12753: Async logging ring buffer and OOM error. When very long messages are written (1M messages or so), + it can produce an OOM error. Log messages are truncated at 10K via configuration in the log4j2.xml files. + +* SOLR-13227: Optimizing facet.range.other by avoiding expensive exceptions (Nikolay Khitrin via Mikhail Khludnev) + +* SOLR-9079: Remove commons-lang as a dependency (Kevin Risden) + +* SOLR-11473: Make HDFSDirectoryFactory support other prefixes (besides hdfs:/) (Kevin Risden) + +* SOLR-13359: Make UpdateHandler support other prefixes (besides hdfs:/) (Kevin Risden) + +* SOLR-13398: Move log line "Processing SSL Credential Provider chain..." from INFO to DEBUG (janhoy) + +* SOLR-13407: Reject update requests sent to non-routed multi collection aliases. (ab) + +* SOLR-11035: (at least) 2 distinct failures possible when clients attempt searches during SolrCore reload, + added test band-aid for DocValuesNotIndexedTest. + +* SOLR-13337: Only request the minimum required number of terms from each shard when using terms.sort=index and none + are discarded due to terms.min/maxcount (Morten Bøgeskov,Munendra S N via Mikhail Khludnev) + +* SOLR-12167: Throw an exception, instead of just a warning, when unknown atomic update operation is + encountered (Munendra S N via Ishan Chattopadhyaya) + +* SOLR-13394: Switch default GC from CMS to G1 (Ishan Chattopadhyaya, Kesharee Nandan Vishwakarma, Shawn Heisey, + Uwe Schindler, Erick Ericsson, shalin) + +* SOLR-13432: Add .toString methods to BitDocSet and SortedIntDocSet so that enabling "showItems" on the filter caches + shows some useful information about the values in the cache. (shalin) + +* SOLR-12833: Avoid unnecessary memory cost when DistributedUpdateProcessor timed-out lock is not used. + (jefferyyuan, ab) + +* SOLR-13348: Speed up collapsing by avoiding scoring of ineligible documents (Andrzej Wislowski via + Ishan Chattopadhyaya) + +Other Changes +---------------------- + +* SOLR-11763: Upgrade Guava to 25.1-jre (Markus Jelsma, Kevin Risden) + +* SOLR-13222: Improve logging in StreamingSolrClients (Peter Cseh via Kevin Risden) + +* SOLR-12055: Enable async logging by default. This change improves throughput for logging. This opens + up a small window where log messages could possibly be lost. If this is unacceptable, switching back to synchronous + logging can be done by changing the log4j2.xml file, no internal Solr code changed to make async logging the default. + (Erick Erickson) + +* SOLR-12753: Async logging ring buffer and OOM error. When very long messages are written (1M messages or so), + it can produce an OOM error. Log messages are truncated at 10K via configuration in the log4j2.xml files. + +* SOLR-9763: Remove the workaround implemented for HADOOP-12767 (Kevin Risden) + +* SOLR-13060: Improve HdfsAutoAddReplicasIntegrationTest and HdfsCollectionsAPIDistributedZkTest (Kevin Risden) + +* SOLR-13074: MoveReplicaHDFSTest leaks threads, falls into an endless loop, logging like crazy (Kevin Risden) + +* SOLR-9762: Remove the workaround implemented for HADOOP-13346 (Kevin Risden) + +* SOLR-7321: Remove reflection in FSHDFSUtils.java (Mike Drob, Kevin Risden) + +* SOLR-13268: Clean up any test failures resulting from defaulting to async logging (Erick Erickson) + +* SOLR-13307: Ensure HDFS tests clear System properties they set (Kevin Risden) + +* SOLR-13330: Improve HDFS tests (Kevin Risden) + +* SOLR-8033: Remove debug if branch in HdfsTransactionLog (Kevin Risden) + +* SOLR-12955: Refactored DistributedUpdateProcessor to put SolrCloud functionality into a subclass. + (Bar Rotstein, David Smiley) + +* SOLR-13342: Remove dom4j from Solr (Kevin Risden) + +* SOLR-13335: Upgrade to velocity 2.0 and velocity-tools 3.0 (Kevin Risden) + +* SOLR-13112: Upgrade jackson to 2.9.8 (Kevin Risden) + +* SOLR-13353: Add SolrCli AuthTool test (Kevin Risden) + +* SOLR-13363: Upgrade to ZooKeeper 3.4.14 (Erick Erickson) + +* SOLR-12809: Document recommended Java/Solr combinations (Erick Erickson, Jan Høydahl et.al.) + +* SOLR-13366: Clarify 'Invalid stage name' warning logging in AutoScalingConfig (Christine Poerschke) + +* SOLR-13409: Disable HTML directory listings in admin interface to prevent possible security issues (Uwe Schindler) + +* SOLR-12461: Upgrade Dropwizard Metrics to 4.0.5 release. (ab) + +* SOLR-13400: Replace Observable pattern in TransientSolrCoreCache (Erick Erickson) + +* SOLR-13425: Ref-Guide: Wrong color of text in left column of horizontal definition list (janhoy) + +* SOLR-13423: Upgrade RRD4j to version 3.5. (ab) + +* SOLR-13414: SolrSchema - Avoid NPE if Luke returns field with no type defined (Kevin Risden) + +================== 8.0.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.19.1 +Carrot2 3.16.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache ZooKeeper 3.4.13 +Jetty 9.4.14.v20181114 + +Upgrade Notes +---------------------- + +* Due to the introduction of LIR redesign since Solr 7.3 (SOLR-11702) and the removing of old LIR implementation in Solr 8. + Rolling updates are not possible unless all nodes must be on Solr 7.3 or higher. If not updates can be lost. + +* Solr nodes can now listen and serve HTTP/2 requests. Most of internal requests (sent by UpdateShardHandler, HttpShardHandler) + Http2SolrClient is used. Since by default, internal requests are sent by using HTTP/2, + Solr 8.0 nodes can't talk to old nodes (7.x). However we can follow these steps to do rolling updates: + - Do rolling updates as normally, but the Solr 8.0 nodes must start with -Dsolr.http1=true as startup parameter. + By using this parameter internal requests are sent by using HTTP/1.1 + - When all nodes are upgraded to 8.0, restart them, this time -Dsolr.http1 parameter should be removed. + +* ALPN is not supported in Java 8 or lower version therefore when a node started in Java 8 or a lower version with SSL enabled, + it will send and can only be able to handle HTTP/1.1 requests. In case of using SSL Java 9 or latter versions are recommended. + +* Custom AuthenticationPlugin must provide its own setup for Http2SolrClient through + implementing HttpClientBuilderPlugin.setup, if not internal requests can't be authenticated. + +* LUCENE-7996: The 'func' query parser now returns scores that are equal to 0 + when a negative value is produced. This change is due to the fact that + Lucene now requires scores to be positive. (Adrien Grand) + +* SOLR-11882: SolrMetric registries retained references to SolrCores when closed. A + change of SolrMetricMAnager.registerGauge and SolrMetricProducer.initializeMetrics + method signatures was required to fix it. Third party components that use this API + need to be updated. (Eros Taborelli, Erick Erickson, ab) + +* LUCENE-8267: Memory codecs have been removed from the codebase (MemoryPostings, + MemoryDocValues). If you used postingsFormat="Memory" or docValuesFormat="Memory" + then either remove it to use the default or experiment with one of the others. (Dawid Weiss) + +* SOLR-12586: The date format patterns used by ParseDateFieldUpdateProcessorFactory (present in "schemaless mode") + are now interpreted by Java 8's java.time.DateTimeFormatter instead of Joda Time. The pattern language is very + similar but not the same. Typically, simply update the pattern by changing an uppercase 'Z' to lowercase 'z' and + that's it. For the current recommended set of patterns in schemaless mode, see "Schemaless Mode" in the ref guide, + or simply examine the default configSet. Also note that the set of patterns (formats) here have + expanded from before to subsume those patterns previously handled by the "extract" contrib (Solr Cell / Tika). + (David Smiley, Bar Rotstein) + +* SOLR-12593: The "extraction" contrib (Solr Cell) no longer does any date parsing, and thus no longer has the + "date.formats" configuration. To ensure date strings are properly parsed, use ParseDateFieldUpdateProcessorFactory + (an URP) commonly registered with the name "parse-date" in "schemaless mode". (David Smiley, Bar Rotstein) + +* SOLR-12643: Since Http2SolrClient does not support exposing connections related metrics. These metrics are no longer + available 'QUERY.httpShardHandler.{availableConnections, leasedConnections, maxConnections, pendingConnections}', + 'UPDATE.updateShardHandler.{availableConnections, leasedConnections, maxConnections, pendingConnections}' + +* SOLR-12605: UpdateShardHandler's updateOnlyClient is now a Http2SolrClient (previous HttpSolrClient). This new + client does not support 'maxConnections','maxConnectionsPerHost' parameters. + +* SOLR-12640: HttpShardHandlerFactory's defaultClient is now a Http2SolrClient (previous HttpSolrClient). This new + client does not support 'maxConnections','maxConnectionsPerHost' parameters. LBHttpSolrClient.Req and LBHttpSolrClient.Rsp + are marked as deprecated, uses LBSolrClient.Req and LBSolrClient.Rsp instead. + +* SOLR-12754: The UnifiedHighlighter hl.weightMatches now defaults to true. If there are unforseen highlight problems, + this may be the culprit. + +* If you explicitly use BM25SimilarityFactory in your schema, the absolute scoring will be lower due to SOLR-13025. + But ordering of documents will not change in the normal case. Use LegacyBM25SimilarityFactory if you need to force + the old 6.x/7.x scoring. Note that if you have not specified any similarity in schema or use the default + SchemaSimilarityFactory, then LegacyBM25Similarity is automatically selected for 'luceneMatchVersion' < 8.0.0. + See also explanation in Reference Guide chapter "Other Schema Elements". + +* SOLR-12535: Solr no longer accepts index time boosts in JSON provided to Solr. This used to be provided like so: + {'id':'1', 'val_s':{'value':'foo', 'boost':2.0}} but will now produce an error. A object/map structure will now only + be interpreted as a child document or an atomic update; nothing else. A uniqueKey is currently required on all child + documents to be interpreted as such, though this may change in the future. (David Smiley) + +* SOLR-12633: When JSON data is sent to Solr with nested child documents split using the "split" parameter, the child + docs will now be associated to their parents by the field/label string used in the JSON instead of anonymously. Most + users probably won't notice the distinction since the label is lost any way unless special fields are in the schema. + This choice used to be toggleable with an internal/expert "anonChildDocs" parameter flag which is now gone. + (David Smiley) + +* SOLR-11774: In 'langid' contrib, the LanguageIdentifierUpdateProcessor base class changed some method signatures. + If you have a custom language identifier implementation you will need to adapt your code. + +* SOLR-9515: Hadoop dependencies have been upgraded to Hadoop 3.2.0 from 2.7.2. (Mark Miller, Kevin Risden) + +* SOLR-5211: Deleting (or updating) documents by their uniqueKey is now scoped to only consider root documents, not + child/nested documents. Thus a delete-by-id won't work on a child doc (no-op), and an attempt to update a child doc + by providing a new doc with the same ID would add a new doc (probably erroneous). Both these actions were and still + are problematic. In-place-updates are safe though. If you want to delete certain child documents and if you know + they don't themselves have nested children then you must do so with a delete-by-query technique. + +* SOLR-13248: The default replica placement strategy used in Solr has been reverted to the 'legacy' policy used by Solr + 7.4 and previous versions. This is due to multiple bugs in the autoscaling based replica placement strategy that was + made default in Solr 7.5 which causes multiple replicas of the same shard to be placed on the same node in addition + to the maxShardsPerNode and createNodeSet parameters being ignored. Although the default has changed, autoscaling + will continue to be used if a cluster policy or preference is specified or a collection level policy is in use. + The default replica placement strategy can be changed to use autoscaling again by setting a cluster property: + curl -X POST -H 'Content-type:application/json' --data-binary ' + { + "set-obj-property": { + "defaults" : { + "cluster": { + "useLegacyReplicaAssignment":false + } + } + } + }' http://$SOLR_HOST:$SOLR_PORT/api/cluster + + +New Features +---------------------- + +* SOLR-12591: Expand the set of recognized date format patterns of schemaless mode to subsume those handled by the + "extract" contrib (Solr Cell / Tika). This is primarily a change in configuration of the default configSet for more + patterns, but also included enabling "lenient" parsing in ParseDateFieldUpdateProcessorFactory. The default + locale was changed from ROOT to en_US since well-known patterns assume this locale. + (David Smiley, Bar Rotstein) + +* SOLR-12879: MinHash query parser that builds queries providing a measure of Jaccard similarity (Andy Hind via Tommaso Teofili) + +* SOLR-12593: The default configSet now includes an "ignored_*" dynamic field. (David Smiley) + +* SOLR-12799: Allow Authentication Plugins to intercept internode requests on a per-request basis. + The BasicAuth plugin now supports a new parameter 'forwardCredentials', and when set to 'true', + user's BasicAuth credentials will be used instead of PKI for client initiated internode requests. (janhoy, noble) + +* SOLR-12791: Add Metrics reporting for AuthenticationPlugin (janhoy) + +* SOLR-12730: Implement staggered SPLITSHARD requests in IndexSizeTrigger. (ab) + +* SOLR-12639: Umbrella JIRA for adding support HTTP/2 (Cao Manh Dat) + +* SOLR-12768, SOLR-13129: Improved nested document support, and enabled in the default schema with the presence of _nest_path_. + When this field is present, certain things happen automatically. An internal URP is automatically used to populate + it. The [child] (doc transformer) will return a hierarchy with relationships; no params needed. The relationship + path is indexed for use in queries (can be disabled if not needed). Also, child documents needn't provide a uniqueKey + value as Solr will supply one automatically by concatenating a path to that of the parent document's key. + (David Smiley, Moshe Bla). + +* SOLR-13134: Allow the knnRegress Stream Evaluator to more easily perform bivariate regression. + (Joel Bernstein) + +* SOLR-13104: Add natural and repeat Stream Evaluators (Joel Bernstein) + +* SOLR-13147: Add movingMAD Stream Evaluator (Joel Bernstein) + +* SOLR-13155: Add command-line option for testing autoscaling configurations. (ab) + +* SOLR-13241: Add 'autoscaling' tool support to solr.cmd (Jason Gerlowski) + +* SOLR-11126: New Node-level health check handler at /admin/info/healthcheck and /node/health paths that + checks if the node is live, connected to zookeeper and not shutdown. (Anshum Gupta, Amrit Sarkar, shalin) + +Bug Fixes +---------------------- + +* SOLR-13058: Fix block that was synchronizing on the wrong collection in OverseerTaskProcessor (Gus Heck) + +* SOLR-11774: langid.map.individual now works together with langid.map.keepOrig. Also the detectLanguage() API + is changed to accept a Reader allowing for more memory efficient implementations (janhoy) + +* SOLR-13126: Query boosts were not being combined correctly for documents where not all boost queries + matched (Alan Woodward, Mikhail Khludnev) + +* SOLR-13248: Autoscaling based replica placement is broken out of the box. Solr 7.5 enabled autoscaling based replica + placement by default but in the absence of default cluster policies, autoscaling can place more than 1 replica of the + same shard on the same node. Also, the maxShardsPerNode and createNodeSet was not respected. Due to these reasons, + this issue reverts the default replica placement policy to the 'legacy' assignment policy that was the default until + Solr 7.4. (Gus Heck, Andrzej Bialecki, Bram Van Dam, shalin) + +* SOLR-13255 : ClasscastException when URPs try to read a String field which returns a ByteArrayUTF8CHarSequence . This is a regression + in release 7.7 (noble) + +*SOLR-13299: Fix Windows startup script to disable HTTP/2 if TLS is enabled on Java 8. (Uwe Schindler) + + +Improvements +---------------------- + +* SOLR-5211: If _root_ is defined in the schema, it is now always populated automatically. This allows documents with + children to be updated with a document that does not have children, whereas before it would break block-join queries. + If you don't use nested documents then _root_ can be removed from the schema. (Dr Oleg Savrasov, Moshe Bla, + David Smiley, Mikhail Khludnev) + +* SOLR-6117: The response format has changed slightly for ReplicationHandler error-cases. All errors now have a non-200 + 'status' field, a 'message' field giving more details on the error, and an optional 'exception' field. (Shalin Mangar, + Jason Gerlowski) + +* SOLR-12888: The NestedUpdateProcessor (which populates internal fields for nested child docs) is now auto-registered + to run immediately prior to RunUpdateProcessor. If the schema has no _nest_*_ fields then it's a no-op. + RunUpdateProcessorFactory looks up a special/internal URP chain "_preRun_" which is implicitly defined but could be + defined by an app for customization if desired. (David Smiley) + +Optimizations +---------------------- + +* SOLR-12725: ParseDateFieldUpdateProcessorFactory should reuse ParsePosition. (ab) + +* SOLR-13025: Due to LUCENE-8563, the BM25Similarity formula no longer includes the (k1+1) factor in the numerator + This gives a lower absolute score but doesn't affect ordering, as this is a constant factor which is the same + for every document. Use LegacyBM25SimilarityFactory if you need the old 6.x/7.x scoring. See also upgrade notes (janhoy) + +* SOLR-13130: during the ResponseBuilder.STAGE_GET_FIELDS directly copy string bytes and avoid creating String Objects (noble) + +Other Changes +---------------------- + +* SOLR-12614: Make "Nodes" view the default in AdminUI "Cloud" tab (janhoy) + +* SOLR-12586: Upgrade ParseDateFieldUpdateProcessorFactory (present in "schemaless mode") to use Java 8's + java.time.DateTimeFormatter instead of Joda time (see upgrade notes). "Lenient" is enabled. Removed Joda Time dependency. + (David Smiley, Bar Rotstein) + +* SOLR-5163: edismax now throws an exception when qf refers to a nonexistent field (Charles Sanders, David Smiley) + +* SOLR-12805: Store previous term (generation) of replica when start recovery process (Cao Manh Dat) + +* SOLR-12652: Remove SolrMetricManager.overridableRegistryName method (Peter Somogyi via David Smiley) + +* LUCENE-8513: SlowCompositeReaderWrapper now uses MultiTerms directly instead of MultiFields (David Smiley) + +* SOLR-11812: Remove backward compatibility of old LIR implementation in 8.0 (Cao Manh Dat) + +* SOLR-12620: Remove the Admin UI Cloud -> Graph (Radial) view (janhoy) + +* SOLR-12775: LowerCaseTokenizer is deprecated, and should be replaced by LetterTokenizer and + LowerCaseFilter (Alan Woodward) + +* SOLR-13036: Fix retry logic in JettySolrRunner (Gus Heck) + +* SOLR-12535: Solr no longer accepts index time boosts in JSON provided to Solr. (David Smiley) + +* SOLR-13086: Improve the error message reported by DocumentObjectBinder when a setter is not found (Gus Heck) + +* SOLR-12365: Renamed class Config to XmlConfigFile (David Smiley) + +* SOLR-9515: Hadoop dependencies have been upgraded to Hadoop 3.2.0 from 2.7.2. (Mark Miller, Kevin Risden) + +================== 7.7.2 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.19.1 +Carrot2 3.16.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache ZooKeeper 3.4.14 +Jetty 9.4.14.v20181114 + +Upgrade Notes +---------------------- + +* SOLR-13234: The Prometheus Metric Exporter now collects metrics from Solr at a fixed schedule that is controlled + by the exporter itself. The default interval is 60s but it can be changed by specifying the --scrape-interval + command line parameter to the tool. The collected metrics are cached and returned for any requests coming in before + the next scheduled collection. This can impact the freshness of the metrics if prometheus is configured to query the + tool more frequently. + +* SOLR-12708: When requesting the status of an async request via REQUESTSTATUS collections API, the response will + include the list of internal async requests (if any) in the "success" or "failed" keys (in addition + to them being included outside those keys for backwards compatibility). See SOLR-12708 for more + details + +Bug fixes +---------------------- + +* SOLR-13234: Prometheus Metric Exporter not threadsafe. This changes the prometheus exporter to collect metrics + from Solr on a fixed interval controlled by this tool and prevents concurrent collections. This change also improves + performance slightly by using the cluster state instead of sending multiple HTTP requests to each node to lookup + all the cores. + (Danyal Prout via shalin) + +* SOLR-12708: Async collection actions should not hide internal failures (Mano Kovacs, Varun Thacker, Tomás Fernández Löbbe) + +* SOLR-11876: In-place update fails when resolving from Tlog if schema has a required field (Justin Deoliveira, janhoy, + Ishan Chattopadhyaya) + +* SOLR-13349: High CPU usage in Solr due to Java 8 bug (Erick Erickson) + +* SOLR-13355: 'all' permission ignored by RuleBasedAuthorizationPlugin in most cases (Jason Gerlowski, janhoy) + +* SOLR-13344: Admin UI inaccessible with RuleBasedAuthorizationPlugin (janhoy, Jason Gerlowski) + +* SOLR-12860: MetricsHistoryHandler now uses PKI Auth for metrics collection in background thread (janhoy, Lorenzo) + +* SOLR-13331: Atomic Update 'remove' operations broken for certain field types in SolrJ (Thomas Wockinger via Jason Gerlowski) + +* SOLR-13285: Updates with enum fields and javabin cause ClassCastException (noble) + +* SOLR-12371: Editing authorization config via REST API now works in standalone mode (janhoy) + +* SOLR-13281: Fixed NPE in DocExpirationUpdateProcessorFactory (Munendra S N, Tomás Fernández Löbbe) + +* SOLR-13409: Disable HTML directory listings in admin interface to prevent possible security issues (Uwe Schindler) + +* SOLR-13126: Query boosts were not being combined correctly for documents where not all boost queries + matched (Alan Woodward, Mikhail Khludnev) + +* SOLR-13414: SolrSchema - Avoid NPE if Luke returns field with no type defined (Kevin Risden) + +* SOLR-12833: Avoid unnecessary memory cost when DistributedUpdateProcessor timed-out lock is not used. (ab) + +* SOLR-13449: SolrClientNodeStateProvider always retries on requesting metrics from other nodes (Cao Manh Dat) + +* SOLR-13112: Upgrade jackson to 2.9.8 (Kevin Risden) + +* SOLR-13410: Designated overseer wasn't able to rejoin election queue at head upon restart (Ishan Chattopadhyaya, + Kesharee Nandan Vishwakarma) + +* SOLR-13254: Correct message that is logged in solrj's ConnectionManager when an exception + occurred while reconnecting to ZooKeeper. (hu xiaodong via Christine Poerschke) + +* SOLR-13252: Fix an NPE when setting a "policy" property for an existing collection (ab) + +* SOLR-13398: Move log line "Processing SSL Credential Provider chain..." from INFO to DEBUG (janhoy) + +* SOLR-13386: OverseerTaskQueue#remove should not throw an exception when no node exists after an exists + check and the Overseer work loop should not allow free spinning the loop when it hits a KeeperException. + (Mark Miller, Fernandez-Lobbe, Mike Drob) + +* SOLR-13366: Clarify 'Invalid stage name' warning logging in AutoScalingConfig (Christine Poerschke) + +* SOLR-13352: Remove risk of deadlock/threadleak when shutting down an Overseer(TriggerThread). (hossman) + +* SOLR-13229: Cleanup replicasMetTragicEvent after all types of exception (Tomás Fernández Löbbe) + +Other Changes +---------------------- + +* SOLR-13363: Upgrade to ZooKeeper 3.4.14 (Erick Erickson) + +================== 7.7.1 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.19.1 +Carrot2 3.16.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache ZooKeeper 3.4.13 +Jetty 9.4.14.v20181114 + +Upgrade Notes +---------------------- + + * SOLR-13248: The default replica placement strategy used in Solr has been reverted to the 'legacy' policy used by Solr + 7.4 and previous versions. This is due to multiple bugs in the autoscaling based replica placement strategy that was + made default in Solr 7.5 which causes multiple replicas of the same shard to be placed on the same node in addition + to the maxShardsPerNode and createNodeSet parameters being ignored. Although the default has changed, autoscaling + will continue to be used if a cluster policy or preference is specified or a collection level policy is in use. + The default replica placement strategy can be changed to use autoscaling again by setting a cluster property: + curl -X POST -H 'Content-type:application/json' --data-binary ' + { + "set-obj-property": { + "defaults" : { + "cluster": { + "useLegacyReplicaAssignment":false + } + } + } + }' http://$SOLR_HOST:$SOLR_PORT/api/cluster + +Bug Fixes +---------------------- + +* SOLR-13255 : ClasscastException when URPs try to read a String field which returns a ByteArrayUTF8CHarSequence . This is a regression + in release 7.7 (noble) + +* SOLR-13248: Autoscaling based replica placement is broken out of the box. Solr 7.5 enabled autoscaling based replica + placement by default but in the absence of default cluster policies, autoscaling can place more than 1 replica of the + same shard on the same node. Also, the maxShardsPerNode and createNodeSet was not respected. Due to these reasons, + this issue reverts the default replica placement policy to the 'legacy' assignment policy that was the default until + Solr 7.4. (Gus Heck, Andrzej Bialecki, Bram Van Dam, shalin) + +================== 7.7.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.19.1 +Carrot2 3.16.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache ZooKeeper 3.4.13 +Jetty 9.4.14.v20181114 + + +Upgrade Notes +---------------------- + +* SOLR-12770: The 'shards' parameter handling logic changes to use a new config element to determine what hosts can be + requested. Please see Apache Solr Reference Guide chapter "Distributed Requests" for details, as well as SOLR-12770. + +New Features +---------------------- + +* SOLR-12839: JSON 'terms' Faceting now supports a 'prelim_sort' option to use when initially selecting + the top ranking buckets, prior to the final 'sort' option used after refinement. (hossman) + +* SOLR-7896: Add a login page to Admin UI, with initial support for Basic Auth (janhoy) + +* SOLR-13116: Add Admin UI login support for Kerberos (janhoy, Jason Gerlowski) + +* SOLR-12770: Make it possible to configure a host whitelist for distributed search + (Christine Poerschke, janhoy, Erick Erickson, Tomás Fernández Löbbe) + +* SOLR-12373: Add a "tombstoneConfig" option to DocBasedVersionConstraintsProcessor that allows + users to configure which fields/values to add to tombstone documents. This can be useful to + make sure tombstone documents include fields that are marked as required in the schema + (Tomás Fernández Löbbe) + +* SOLR-12984: The search Streaming Expression should properly support and push down paging + when using the /select handler (Joel Bernstein) + +* SOLR-13088: Add zplot Stream Evaluator to plot math expressions in Apache Zeppelin (Joel Bernstein) + +Bug Fixes +---------------------- + +* SOLR-12546: CVSResponseWriter omits useDocValuesAsStored=true field when fl=* + (Munendra S N via Mikhail Khludnev) + +* SOLR-12933: Fix SolrCloud distributed commit. (Mark Miller) + +* SOLR-13014: URI Too Long with large streaming expressions in SolrJ (janhoy) + +* SOLR-13066: A failure while reloading a SolrCore can result in the SolrCore not being closed. (Mark Miller) + +* SOLR-11296: Spellcheck parameters not working in new UI (Matt Pearce via janhoy) + +* SOLR-10975: New Admin UI Query does not URL-encode the query produced in the URL box (janhoy) + +* SOLR-13072: Management of markers for nodeLost / nodeAdded events is broken. This bug could have caused + some events to be lost if they coincided with an Overseer leader crash. (ab) + +* SOLR-13080: The "terms" QParser's "automaton" method semi-required that the input terms/IDs be sorted. This + query parser now does this. Unclear if this is a perf issue or actual bug. (Daniel Lowe, David Smiley) + +* SOLR-13082: A trigger that creates trigger events more frequently than the cool down period can starve other triggers. + This is mitigated to some extent by randomly choosing the trigger to resume after cool down. It is recommended that + scheduled triggers not be used for very frequent operations to avoid this problem. + (ab, shalin) + +* SOLR-12514: Rule-base Authorization plugin skips authorization if querying node does not have collection replica (noble) + +* SOLR-11853: Solr installer fails on SuSE linux (Markus Mandalka via janhoy) + +* SOLR-12237: Fix incorrect SOLR_SSL_KEYSTORE_TYPE variable in solr start script (janhoy, Joel Bernstein) + +* SOLR-13053: NodeAddedTrigger and NodeLostTrigger do not reserve added/removed time populated by restoreState + (Cao Manh Dat) + +* SOLR-13137: NPE when /admin/zookeeper/status endpoint hit in standalone mode (janhoy) + +* SOLR-13091: REBALANCELEADERS is broken (Erick Erickson) + +* SOLR-11998: RebalanceLeaders API broken response format with wt=JSON (Erick Erickson) + +* SOLR-9735: Fix v2 API for AutoscalingHistoryHandler. (ab) + +* SOLR-13168: Fixed a bug in TestInjection that caused test only code to be invoked when TLOG replicas + recieved commits if java assertions were enabled. (hossman) + +Improvements +---------------------- + +* SOLR-12881: Remove unneeded import statements (Peter Somogyi via Erick Erickson) + +* SOLR-12992: When using binary format, ExportWriter to directly copy BytesRef instead of + creating new String (noble) + +* SOLR-12898: Replace cluster state polling with ZkStateReader#waitFor. (Mark Miller) + +* SOLR-12897: Introduce AlreadyClosedException to clean up silly close / shutdown logging. (Mark Miller) + +* SOLR-12896: Introduce more checks for shutdown and closed to improve clean close and shutdown. (Mark Miller) + +* SOLR-12804: Remove static modifier from Overseer queue access. (Mark Miller) + +* SOLR-12833: Add configurable timeout to VersionBucket lock. (Jeffery Yuan, Mark Miller) + +* SOLR-12885: BinaryResponseWriter (javabin format) should directly copy from BytesRef to output (noble) + +* SOLR-12973: Admin UI "Nodes" view support for replica* replica names. (Daniel Collins, Christine Poerschke, janhoy) + +* SOLR-13090: All shipped configurations still have `maxBooleanClauses` default to 1024. But if the + `solr.max.booleanClauses` sysprop is specified, that will override the 1024 default. This enables users to + update this property across the board more easily. (Jason Gerlowski) + +* SOLR-12983: JavabinLoader should avoid creating String Objects and create UTF8CharSequence fields from byte[] (noble) + +* SOLR-13016: Computing suggestions when policy have "#EQUAL" or "#ALL" rules take too long (noble) + +* SOLR-13029: solr.hdfs.buffer.size can be configured for HdfsBackupRepository for better performance (Tim Owen via Mikhail Khludnev) + +* SOLR-13156: support facet.sort for facet.field={!terms=foo,bar}field. (Konstantin Perikov via Mikhail Khludnev) + +* SOLR-13146: Allow derivatives to be computed for the oscillate Stream Evaluator (Joel Bernstein) + +Other Changes +---------------------- + +* SOLR-12972: deprecate unused SolrIndexConfig.luceneVersion (Christine Poerschke) + +* SOLR-12801: Make massive improvements to the tests. (Mark Miller) + +* SOLR-12923: The new AutoScaling tests are way too flaky and need special attention. (Mark Miller) + +* SOLR-12932: ant test (without badapples=false) should pass easily for developers. (Mark Miller) + +* SOLR-13036: Fix retry logic in JettySolrRunner (Gus Heck) + +* SOLR-12727: Upgrade ZooKeeper dependency to 3.4.13 (Kevin Risden, Erick Erickson, Cao Manh Dat) + +* SOLR-13086: Improve the error message reported by DocumentObjectBinder when a setter is not found (Gus Heck) + + +================== 7.6.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.19.1 +Carrot2 3.16.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache ZooKeeper 3.4.11 +Jetty 9.4.11.v20180605 + +Upgrade Notes +---------------------- + +* SOLR-12767: The min_rf parameter is no longer needed, Solr will always return the achieved replication factor (rf) + in the response header. + +* SOLR-12827: The cluster wide defaults structure has changed from {collectionDefaults: {nrtReplicas : 2}} to + {defaults : {collection : {nrtReplicas : 2}}}. The old format continues to be supported and can be read from + ZK as well as written using the V2 set-obj-property syntax but it is deprecated and will be removed in Solr 9. + We recommend that users change their API calls to use the new format going forward. + +* SOLR-12739: Autoscaling policy framework is now used as the default strategy to select the nodes on which + new replicas or replicas of new collections are created. Previously, the maxShardsPerNode parameter was not allowed + on collections when autoscaling policy was configured. Also if an autoscaling policy was configured then the default + was to set an unlimited maxShardsPerNode automatically. Now the maxShardsPerNode parameter is always + allowed during collection creation and maxShardsPerNode should be set correctly (if required) regardless of whether + autoscaling policies are in effect or not. The default value of maxShardsPerNode continues to be 1 as before. It can + be set to -1 during collection creation to fall back to the old behavior of unlimited maxShardsPerNode when using + autoscaling policy. + +* SOLR-12861: Added a Solr factory for ByteBuffersDirectory, which will replace deprecated RAMDirectory in Solr 9.0. + +New Features +---------------------- + +* SOLR-6280: {!collapse}: if you attempt to use CollapseQParser on a field that is multi-valued, you will now get an + error. Previously, the collapsing behavior was unreliable and undefined despite no explicit error. + (Munendra S N, David Smiley) + +* SOLR-9317: ADDREPLICA command should be able to add more than one replica to a collection,shard at a time. + The API now supports 'nrtReplicas', 'tlogReplicas', 'pullReplicas' parameters as well 'createNodeSet' parameter. + As part of this change, the CREATESHARD API now delegates placing replicas entirely to the ADDREPLICA command + and uses the new parameters to add all the replicas in one API call. (shalin) + +* SOLR-11522: /autoscaling/suggestions now include rebalance options as well even if there are no violations (noble) + +* SOLR-12822: /autoscaling/suggestions to include suggestion to add-replica for lost replicas (noble) + +* SOLR-12815: Implement maxOps limit for IndexSizeTrigger. (ab) + +* SOLR-12843: Implement a MultiContentWriter in SolrJ to post multiple files/payload at once (noble) + +* SOLR-12780: Add support for Leaky ReLU and TanH activations in contrib/ltr NeuralNetworkModel class. + (Kamuela Lau, Christine Poerschke) + +* SOLR-12846: Added support for "host" variable in autoscaling policy rules (noble) + +* SOLR-5004: Splitshard collections API now supports splitting into more than 2 sub-shards directly i.e. by providing a + numSubShards parameter (Christine Poerschke, Anshum Gupta) + +* SOLR-12754: The UnifiedHighlighter has a new hl.weightMatches param defaulting to false (will be true in 8.0). It's + the highest query accuracy mode, and furthermore phrase queries are highlighted as one. (David Smiley) + +* SOLR-12828: Add oscillate Stream Evaluator to support sine wave analysis. (Joel Bernstein) + +* SOLR-11907: Add convexHull and associated geometric Stream Evaluators. (Joel Bernstein) + +* SOLR-12811: Add enclosingDisk and associated geometric Stream Evaluators. (Joel Bernstein) + +* SOLR-12840: Add pairSort Stream Evaluator (Joel Bernstein) + +* SOLR-12862: Add log10 Stream Evaluator and allow the pow Stream Evaluator to accept a vector of exponents (Joel Bernstein) + +* SOLR-12942: Add an option in IndexSizeTrigger to select the split shard method. (ab) + +* SOLR-12938: Cluster Status returns results for aliases, instead of throwing exceptions (Gus Heck) + +* SOLR-11997: Suggestions API/UI should show an entry where a violation could not be resolved (noble) + +* SOLR-12971: Add pivot Stream Evaluator to pivot facet co-occurrence counts into a matrix (Joel Bernstein) + +* SOLR-12795: Introduce 'rows' and 'offset' parameter in FacetStream (Joel Bernstein, Amrit Sarkar, Varun Thacker) + +* SOLR-11572: Add recip Stream Evaluator to support reciprocal transformations (Joel Bernstein) + +* SOLR-12936: Allow percentiles Stream Evaluator to accept an array of percentiles to calculate (Joel bernstein) + +* SOLR-12829: Add plist (parallel list) Streaming Expression (Joel Bernstein) + +* SOLR-12975: Add ltrim and rtrim Stream Evaluators (Joel Bernstein) + +* SOLR-12962: Added a new 'uninvertible' option for fields and fieldtypes. This defaults to 'true' for + backcompat allowing a FieldCache to be built for indexed fields as needed, but users are encouraged + to set this to false (using docValues as needed) to reduce the risk of large fluxuations in heap + size due to unexpected attempts to sort/facet/function on non-docValue fields. (hossman) + + +Other Changes +---------------------- + +* SOLR-12762: Fix javadoc for SolrCloudTestCase.clusterShape() method and add a method that validates only against + Active slices (Anshum Gupta) + +* SOLR-12756: Refactor Assign and extract replica placement strategies out of it. Now, assignment is done with the help + of a builder class instead of calling a method with large number of arguments. The number of special cases that had + to be handled have been cut down as well. (shalin) + +* SOLR-12827: Migrate cluster wide defaults syntax in cluster properties to a nested structure. The structure has + changed from {collectionDefaults: {nrtReplicas : 2}} to {defaults : {collection : {nrtReplicas : 2}}}. + (ab, shalin) + +* SOLR-12835: Document statistics exposed by the Query Result Cache when maxRamMB is configured. (shalin) + +* SOLR-12423: Upgrade to Tika 1.19.1 when available (Tim Allison via Erick Erickson) + +* SOLR-12793: Move TestCloudJSONFacetJoinDomain and TestCloudJSONFacetSKG to the facet test package (Varun Thacker) + +* SOLR-12861: Add Solr factory for ByteBuffersDirectory. + +* SOLR-12746: Simplify the Ref Guide HTML structure and use semantic HTML tags where possible. Adds new template files + for Asciidoctor HTML conversion. Building the HTML version now requires the Slim gem. (Cassandra Targett) + +* SOLR-12956: Add Javadoc @since tag to Analyzer component classes (Alexandre Rafalovitch) + +* SOLR-12966: Add Javadoc @since tag to URP classes (Alexandre Rafalovitch) + +* SOLR-12600: Fix parameter names in Solr JSON documentation (Alexandre Rafalovitch) + +* SOLR-12497: Add documentation to use Hadoop credential provider-based keystore/trustsore. +(Mano Kovacs, Cassandra Targett) + +* SOLR-13006: ZkNodeProps to be able to load from both javabin and JSON (noble) + +Bug Fixes +---------------------- + +* SOLR-12803: Ensure ConcurrentUpdateSolrClient sends documents to correct collection (Jason Gerlowski) + +* SOLR-11836: FacetStream works with bucketSizeLimit of -1 which will fetch all the buckets. + (Alfonso Muñoz-Pomer Fuentes, Amrit Sarkar via Varun Thacker) + +* SOLR-12776: Setting of TMP in solr.cmd causes invisibility of Solr to JDK tools (Petr Bodnar via Erick Erickson) + +* SOLR-12648: Autoscaling framework based replica placement is not used unless a policy is specified or + non-empty cluster policy exists. (shalin) + +* SOLR-12750: Migrate API should lock the collection instead of shard. (shalin) + +* SOLR-12767: When using min_rf, shard leader skipped bad replicas instead of marking them for recovery, this could + create inconsistencies between replicas of the same shard. min_rf parameter is now deprecated, and even if provided + replicas that don't ack an update from the leader will be marked for recovery. (Tomás Fernández Löbbe) + +* SOLR-12814: Metrics history causing "HttpParser URI is too large >8192" when many collections (janhoy) + +* SOLR-12836: ZkController creates a cloud solr client with no connection or read timeouts. Now the http client + created by the update shard handler is used instead. (shalin) + +* SOLR-12729: SplitShardCmd should lock the parent shard to prevent parallel splitting requests. (ab) + +* SOLR-12851: Improvements and fixes to let and select Streaming Expressions (Joel Bernstein) + +* SOLR-12874: Java 9+ GC Logging filesize parameter should use a unit. (Tim Underwood via Uwe Schindler) + +* SOLR-12868: Request forwarding for v2 API is broken (noble) + +* SOLR-7557: Fix parsing of child documents using queryAndStreamResponse (Marvin Bredal Lillehaug/Stian Østerhaug via janhoy) + +* SOLR-12875: fix ArrayIndexOutOfBoundsException when unique(field) or uniqueBlock(_root_) is +used with DVHASH method in json.facet. (Tim Underwood via Mikhail Khludnev) + +* SOLR-12954: fix facet.pivot refinement bugs when using facet.sort=index and facet.mincount>1 (hossman) + +* SOLR-12023: Autoscaling policy engine shuffles replicas needlessly (noble) + +* SOLR-12243: Edismax missing phrase queries when phrases contain multiterm synonyms + (Elizabeth Haubert, Alessandro Benedetti, Uwe Schindler, Steve Rowe) + +* SOLR-12977: Autoscaling tries to fetch metrics from dead nodes (noble) + +* SOLR-12978: In autoscaling NPE thrown for nodes where value is absent (noble) + +Improvements +---------------------- + +* SOLR-12767: Solr now always includes in the response of update requests the achieved replication factor + (Tomás Fernández Löbbe) + +* SOLR-12782: UninvertingReader wrapping is now a bit more lightweight: Does not create FieldInfo for fields that + can't be uninverted (saves mem) and can avoid wrapping the reader altogether if there's nothing to uninvert. + IndexSchema.getUninversionMap refactored to getUninversionMapper and no longer merges FieldInfos. (David Smiley) + +* SOLR-12739: Make autoscaling policy based replica placement the default strategy for placing replicas. (shalin) + +* SOLR-12806: use autoscaling policies with strict=false to prioritize node allocation (noble) + +* SOLR-10981: Support for stream.url or stream.file pointing to gzipped data. It's detected by either a content + encoding header or file extension. (Andrew Lundgren via David Smiley, Jan Høydahl) + +* SOLR-12892: MapWriter to use CharSequence instead of String (noble) + +* SOLR-12882: Eliminate excessive lambda allocation in json facets FacetFieldProcessorByHashDV (Tim Underwood) + +* SOLR-12699: Make contrib/ltr LTRScoringModel immutable and cache its hashCode. + (Stanislav Livotov, Edward Ribeiro, Christine Poerschke) + +* LUCENE-8557: Some internal LeafReader.getFieldInfos implementations were being re-computed on-demand instead of + once up front leading to some slowdowns in places like JSON Facets and field collapsing. (Tim Underwood, David Smiley) + +* SOLR-12964: Json Facets: use DocValuesIterator advanceExact() instead of advance() in FacetFieldProcessorByHashDV and + UniqueSinglevaluedSlotAcc. (Tim Underwood) + +* SOLR-12880: Json Facets: Show the FacetProcessor class name instead of the FacetRequest in the JSON Facets debug-trace + output (Tim Underwood) + +================== 7.5.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.18 +Carrot2 3.16.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache ZooKeeper 3.4.11 +Jetty 9.4.11.v20180605 + +Upgrade Notes +---------------------- + +* The data-driven schema no longer returns the *_str field-copy of text fields by default. The *_str field is still + created and can be used for e.g. sorting, but to retrieve it you now need to explicitly list it in "fl" (SOLR-12350) + +* LUCENE-7976: TieredMergePolicy now respects maxSegmentSizeMB by default when executing + findForcedMerges (optimize) and findForcedDeletesMerges (expungeDeletes) (Erick Erickson) + +* SOLR-12395: SignificantTermsQParserPlugin's name is now 'significantTerms' and its old name 'sigificantTerms' is deprecated. + +* SOLR-11694: Extremely outdated UIMA contrib module has been removed (Alexandre Rafalovitch) + +* SOLR-12008: The configuration file for log4j2.xml is now in ../server/resources/log4j2.xml. All other copies have been removed. + WINDOWS USERS: This JIRA corrects a bug in the start scripts that put example logs under ...\server, solr.log is + now under ...\example. (Erick Erickson) + +* SOLR-12398: The heatmap facet response structure, when returned in JSON, changed from an even/odd name/value array to + an object style. Although the object style makes more sense, this was an overlooked back-compat break; sorry. + + + +New Features +---------------------- + +* SOLR-11865: The QueryElevationComponent now has a useConfiguredElevatedOrder setting. When multiple docs are elevated, + this specifies whether their relative order should be the order in the configuration file or if not then should they + be subject to whatever the sort criteria is. Additionally, QEC was extensively refactored to be more extensible. + (Bruno Roustant, David Smiley) + +* SOLR-12474: Add an UpdateRequest Object that implements RequestWriter.ContentWriter (noble) + +* SOLR-12361: Allow nested child documents to be in field values of a SolrInputDocument as an alternative to + add/get ChildDocuments off to the side. The latter is now referred to as "anonymous" child documents as opposed to + "labelled" (by the field name). Anonymous child docs might be deprecated in the future. This is an internal change + that should work for javabin/SolrJ; separate issues will address XML & JSON formats populating nested docs in this way. + AddUpdateCommand and it's relationship with DirectUpdateHandler2 was reworked substantially. (Moshe Bla, David Smiley) + +* SOLR-12362: Uploading docs in JSON now supports child documents as field values, thus providing a label to the + relationship instead of the current "anonymous" relationship. Use of this experimental feature sometimes requires a + anonChildDocs=false parameter until Solr 8 due to syntax ambiguities. (Moshe Bla, David Smiley) + +* SOLR-12485: Uploading docs in XML now supports child documents as field values, thus providing a label to the + relationship instead of the current "anonymous" relationship. (Moshe Bla, David Smiley) + +* SOLR-12441: (EXPERIMENTAL) New NestedUpdateProcessorFactory (URP) to populate special fields _nest_parent_ and + _nest_path_ of nested (child) documents. It will generate a uniqueKey of nested docs if they were blank too. + (Moshe Bla, David Smiley) + +* SOLR-12519: The [child] transformer now returns a nested child doc structure (attached as fields if provided this way) + provided the schema has the _nest_path_ field. This is part of a broader enhancement of nested docs. + (Moshe Bla, David Smiley) + +* SOLR-12722: The [child] transformer now takes an 'fl' param to specify which fields to return. It will evaluate + doc transformers if present. In 7.5 a missing 'fl' defaults to the current behavior of all fields, but in 8.0 + defaults to the top/request "fl". (Moshe Bla, David Smiley) + +* SOLR-11578: Solr 7 Admin UI (Cloud > Graph) should reflect the Replica type to give a more accurate representation + of the cluster. (Rhoit Singh via Erick Erickson) + +* SOLR-12507: Modify collection API should support un-setting properties. (shalin) + +* SOLR-12506: Add SolrJ support for the modify collection API. (shalin) + +* SOLR-12398: The JSON Facet API now supports type=heatmap facets, just as classic faceting does. (David Smiley) + +* SOLR-11985: Support percentage values in replica attribute in autoscaling policy (noble) + +* SOLR-12511: Support non integer values for replica in autoscaling policy (noble) + +* SOLR-12517: Support range values for replica in autoscaling policy (noble) + +* SOLR-12530: Ability to disable configset upload via -Dconfigset.upload.enabled=false startup parameter + (Ishan Chattopadhyaya) + +* SOLR-12495: An #EQUAL function for replica in autoscaling policy to equally distribute replicas (noble) + +* SOLR-11986: Allow percentage in freedisk attribute in autoscaling policy rules (noble) + +* SOLR-12522: Support a runtime function `#ALL` for 'replica' in autoscaling policies (noble) + +* SOLR-12567: JSON Facet "functions" now support an extended "type:func" syntax, similar to other types + of facets. This also allows additional local params to be specified for if the aggregation function + can take advantage of them. (hossman) + +* SOLR-12581: the JSON Facet 'relatedness()' aggregate function now supports a 'min_popularity' option + using the extended type:func syntax (hossman) + +* SOLR-12536: autoscaling policy support to equally distribute replicas on the basis of arbitrary properties (noble) + +* SOLR-8207: Add "Nodes" view to the Admin UI "Cloud" tab, listing nodes and key metrics (janhoy) + +* SOLR-7767: "ZK Status" sub menu under "Cloud" tab to see status of zookeeper ensemble (janhoy) + +* SOLR-11990: Make it possible to co-locate replicas of multiple collections together in a node. A collection may be + co-located with another collection during collection creation time by specifying a 'withCollection' parameter. It can + also be co-located afterwards by using the modify collection API. The co-location guarantee is enforced regardless of + future cluster operations whether they are invoked manually via the Collection API or by the Autoscaling framework. + (noble, shalin) + +* SOLR-12402: Factor out SolrDefaultStreamFactory class. (Christine Poerschke) + +* SOLR-12592: support #EQUAL function, range operator, decimal and percentage in cores in autoscaling policies (noble) + +* SOLR-12655: Add Korean morphological analyzer ("nori") to default distribution. This also adds examples + for configuration in Solr's schema. (Uwe Schindler) + +* SOLR-11863: Add knnRegress Stream Evaluator to support nearest neighbor regression (Joel Bernstein) + +* SOLR-12702: Add zscores Stream Evaluator (Joel Bernstein) + +* SOLR-12687: Add functions to cache data structures and mathematical models (Joel Bernstein) + +* SOLR-12671: Add robust flag to knnRegress Stream Evaluator (Joel Bernstein) + +* SOLR-12660: Add outliers Stream Evaluator to support outlier detection with probability distributions + (Joel Bernstein) + +* SOLR-12634: Add gaussfit Stream Evaluator (Joel Bernstein) + +* SOLR-12629: The predict evaluator should work with the polyfit function (Joel Bernstein) + +* SOLR-12715: NodeAddedTrigger should support adding replicas to new nodes by setting preferredOperation=addreplica. + (shalin) + +* SOLR-11861: When creating a configSet via the API, the "baseConfigSet" parameter now defaults to "_default". + (Amrit Sarkar, David Smiley) + +* SOLR-12716: NodeLostTrigger should support deleting replicas from lost nodes by setting preferredOperation=deletenode. + (shalin) + +* SOLR-9418: Added a new (experimental) PhrasesIdentificationComponent for identifying potential phrases + in query input based on overlapping shingles in the index. (Akash Mehta, Trey Grainger, hossman) + +* SOLR-11943: Add machine learning functions for location data (Joel Bernstein) + +* SOLR-12612: Cluster properties restriction of known keys only is relaxed, and now unknown properties starting with "ext." + will be allowed. This allows custom to plugins set their own cluster properties. (Jeffery Yuan via Tomás Fernández Löbbe) + +* SOLR-12357: Time Routed Aliases now have a preemptiveCreateMath option to preemptively and asynchronously create the + next collection in advance as new data gets within this time window of the end. (Gus Heck, David Smiley) + +Bug Fixes +---------------------- + +* SOLR-12449: Response /autoscaling/diagnostics shows improper json (noble) + +* SOLR-11676: Keep nrtReplicas and replicationFactor in sync while creating a collection and modifying a collection + (Varun Thacker) + +* SOLR-12489: User specified replicationFactor and maxShardsPerNode is used when specified during a restore operation. + A user can now specify nrtReplicas/tlogReplicas/pullReplicas while restoring the collection. + Specifying replicationFactor or nrtReplicas have the same effect and only one can be specified (Varun Thacker) + +* SOLR-11216: Race condition in PeerSync (Cao Manh Dat) + +* SOLR-11807: Restoring collection now treats maxShardsPerNode=-1 as unlimited (Varun Thacker) + +* SOLR-12413: If Zookeeper was pre-loaded with data before first-use, then the aliases information would be ignored. + (David Smiley, Gaël Jourdan, Gus Heck) + +* SOLR-12482: Config API returns status 0 for failed operations. (Steve Rowe) + +* SOLR-12513: Reproducing TestCodecSupport.testMixedCompressionMode failure (Erick Erickson) + +* SOLR-11665: Improve error handling of shard splitting. Fix splitting of mixed replica types. (ab) + +* SOLR-12326: JSON Facet API: terms facet shard requests now indicate if they have more buckets to prevent + unnecessary refinement requests. (yonk) + +* SOLR-12427: Improve error message for invalid 'start', 'rows' parameters. (Munendra S N via Jason Gerlowski) + +* SOLR-12395: Make 'significantTerms' the SignificantTermsQParserPlugin's name and deprecate its old 'sigificantTerms' name. + (Tobias Kässmann, Christine Poerschke) + +* SOLR-12533 Collection collection fails if metrics are called during core creation (Peter Cseh, Mano Kovacs) + +* SOLR-2834: Fix SolrJ Field and Document analyzes for types that include CharacterFilter (Alexandre Rafalovitch) + +* SOLR-12516: Fix some bugs in 'type:range' Facet refinement when sub-facets are combined with non + default values for the 'other' and 'include' options. (hossman) + +* SOLR-12343: Fixed a bug in JSON Faceting that could cause incorrect counts/stats when using non default + sort options. This also adds a new configurable "overrefine" option. (Yonik Seeley, hossman) + +* SOLR-12553: Allow SignificantTerms Query Parser to use local parameters (Alexandre Rafalovitch) + +* SOLR-12570: OpenNLPExtractNamedEntitiesUpdateProcessor cannot support multi fields because pattern replacement + doesn't work correctly. (Koji Sekiguchi) + +* SOLR-12576: Update ref guide for additional information displayed in cloud view (Erick Erickson) + +* SOLR-12597: Migrate API should fail requests that do not specify split.key parameter (shalin) + +* SOLR-12477: An update would return a client error(400) if it hit a AlreadyClosedException. + We now return the error as a server error(500) instead (Jeffery via Varun Thacker) + +* SOLR-12606: Fix InfixSuggestersTest.testShutdownDuringBuild() failures. (Steve Rowe) + +* SOLR-12607: Fixed two separate bugs in shard splits which can cause data loss. The first case is when using TLOG + replicas only, the updates forwarded from parent shard leader to the sub-shard leader are written only in tlog and + not the index. If this happens after the buffered updates have been replayed then the updates can never be executed + even though they remain the transaction log. The second case is when synchronously forwarding updates to sub-shard + leader fails and the underlying errors are not propagated to the client. (Cao Manh Dat, shalin) + +* SOLR-12344: SolrSlf4jReporter doesn't set MDC context. (ab) + +* SOLR-12594: MetricsHistoryHandler.getOverseerLeader fails when hostname contains hyphen. (ab) + +* SOLR-12615: HashQParserPlugin will no longer throw an NPE if the hash key field is a string when there are documents + with empty values. All documents with empty values ( string , numeric ) will be processed by worker=0 + This would fix the NPE when using the search stream with partitionKeys. (Varun Thacker) + +* SOLR-11770: NPE in tvrh if no field is specified and document doesn't contain any fields with term vectors + +* SOLR-12541: Metrics handler throws an error if there are transient cores. (ab) + +* SOLR-12470: Search Rate Trigger multiple bug fixes, improvements and documentation updates. (ab) + +* SOLR-12665: Autoscaling policy not being refreshed due to caching (noble) + +* SOLR-12649: CloudSolrClient retries requests unnecessarily exception from server (noble, shalin) + +* SOLR-12670: RecoveryStrategy logs wrong wait time when retrying recovery. (shalin) + +* SOLR-12668: Autoscaling trigger listeners should be executed in the order of their creation. (ab) + +* SOLR-12475: Fix MaxSizeAutoCommitTest failures (Rupa Shankar, Anshum Gupta) + +* SOLR-12674: RollupStream should not use the HashQueryParser for 1 worker. (Varun Thacker) + +* SOLR-12679: MiniSolrCloudCluster internal jetty list should never have duplicates (shalin) + +* SOLR-12598: Do not fetch non-stored fields (Nikolay Khitrin, Erick Erickson) + +* SOLR-12683: HashQuery will throw an exception if more than 4 partitionKeys is specified. + Earlier after the 4th partitionKey the keys would be silently ignored. (Varun Thacker) + +* SOLR-10028: Fix and improvements to SegmentsInfoRequestHandlerTest (Christine Poerschke, Tomás Fernández Löbbe) + +* SOLR-11585: Solr SQL does not work with point numeric fields (Joel Bernstein, Kiran Chitturi) + +* SOLR-12704: Guard AddSchemaFieldsUpdateProcessorFactory against null field names and field values. + (Steve Rowe, Varun Thacker) + +* SOLR-12733: SolrMetricReporterTest failure (Erick Erickson, David Smiley) + +* SOLR-12765: Incorrect format of JMX cache stats. (Bojan Smid, ab) + +* SOLR-12749: timeseries() expression missing sum() results for empty buckets (Joel Bernstein) + +Optimizations +---------------------- + +* SOLR-12350: Do not use docValues as stored for _str (copy)fields in _default configset (janhoy) + +* SOLR-12455: Refactor JSON serialization code into SolrJ package (noble) + +* SOLR-11654: Time Routed Alias will now route documents to the ideal shard of a collection, thus avoiding a hop. + Usually documents were already routed well but not always. (Gus Heck, David Smiley) + +* SOLR-11598: The export handler does not limit users to 4 sort fields and is now unlimited. However the speed at + which we can export is directly proportional to the number of sort fields specified. This change also allows streaming + expressions to group by on more than 4 fields. (Aroop Ganguly, Amrit Sarkar, Varun Thacker) + +* SOLR-12305: When a replica is applying updates, some kind of updates can skip buffering for faster recovery. + (Cao Manh Dat) + +* SOLR-12509: Improve SplitShardCmd performance and reliability. A new method of splitting has been + introduced (splitMethod=link) which uses hard-linking of index files when possible, resulting in + significant speedups and reduced CPU / IO load on shard leader. (ab) + +* SOLR-11881: Retry update requests sent by leaders to it's followers (Varun Thacker, Mark Miller, Tomás Fernández Löbbe) + +* SOLR-12616: Optimize Export writer upto 4 sort fields to get better performance. + This was removed in SOLR-11598 but brought back in the same version (Amrit Sarkar, Varun Thacker) + +* SOLR-12572: While exporting documents using the export writer, if a field is specified as a sort parameter and also + in the fl (field list) parameter, we save on one doc-value lookup. This can bring performance improvements of 15% + and upwards depending on how many fields are in common. (Amrit Sarkar, Varun Thacker) + +* SOLR-10697: HttpShardHandler now uses a default of 100k as maxConnections (10k previously) and default + maxConnectionsPerHost as 100k (20 previously). They are now consisent with the UpdateShardHandler defaults. + (Varun Thacker) + +* SOLR-12723: Reduce object creation in HashBasedRouter. (ab) + +* SOLR-12766: When retrying internal requests, backoff only once for the full batch of retries (Tomás Fernández Löbbe) + +Other Changes +---------------------- + +* SOLR-12208: Renamed the autoscaling variable 'INDEX.sizeInBytes' to 'INDEX.sizeInGB' (noble) + +* SOLR-12523: Improve error reporting and docs regarding Collection backup feature shared-fs requirement (janhoy) + +* SOLR-12468: Upgrade Jetty to 9.4.11.v20180605 (Michael Braun, shalin) + +* SOLR-12527: factor out a test-framework/ConfigRequest class (Christine Poerschke) + +* SOLR-12412: Leader should give up leadership when IndexWriter.tragedy occur (Cao Manh Dat, Tomas Fernandez-Lobbe) + +* SOLR-12551: Upgrade to Tika 1.18 (Tim Allison via Erick Erickson) + +* SOLR-12464: Reduce Overseer.close() logging (for non-Overseer leaders) (Christine Poerschke) + +* SOLR-12454: Tweak Overseer leadership transition related logging for easier troubleshooting. (Christine Poerschke) + +* SOLR-12574: Put under a common "significantTerms" bucket all output by SignificantTerms Query Parser (Alexandre Rafalovitch) + +* SOLR-12164: Improve Ref Guide main landing page. (Cassandra Targett) + +* SOLR-10984: Clean up web.xml, removing old redirects and outdated comments (Varun Thacker, janhoy) + +* SOLR-12617: Remove Commons BeanUtils as a dependency (Varun Thacker) + +* SOLR-11008: Use a lighter config for MetricsHandlerTest and ensure the core is up before the test starts (Varun Thacker) + +* SOLR-11766: Move Streaming Expressions section in Ref Guide to be a top-level section. (Cassandra Targett) + +* SOLR-12656: ShardSplitTest should extend AbstractFullDistribZkTestBase instead of BasicDistributedZkTest. (shalin) + +* LUCENE-8456: Upgrade Apache Commons Compress to v1.18 (Steve Rowe) + +* SOLR-12014: Cryptic error message when creating a collection with sharding that violates autoscaling policies (noble) + +* SOLR-12680: Fix ClassCastException and AIOOBE in TestSolrConfigHandlerConcurrent. (shalin) + +* SOLR-12675: Make LeaderVoteWaitTimeoutTest more resilient against side effects of test methods. (shalin) + +* SOLR-12130: CdcrReplicationDistributedZkTest is broken into two test classes, CdcrOpsAndBoundariesTest which does + not require node restarts and CdcrWithNodesRestartsTest which does. The tests themselves are made faster and more + resilient to spurious failures. (Varun Thacker, Amrit Sarkar via shalin) + +* SOLR-12625: Combine SolrDocumentFetcher and RetrieveFieldsOptimizer (Erick Erickson) + +* SOLR-12690: Regularize LoggerFactory declarations (Erick Erickson) + +* SOLR-12590: Improve Solr resource loader coverage in the ref guide. + (Steve Rowe, Cassandra Targett, Christine Poerschke) + +* SOLR-12744: Improve logging messages and verbosity around recoveries (Cao Manh Dat, Varun Thacker) + +* SOLR-8742: In HdfsDirectoryTest replace RAMDirectory usages with ByteBuffersDirectory. + (hossman, Mark Miller, Andrzej Bialecki, Steve Rowe) + +* SOLR-12771: Improve Autoscaling Policy and Preferences documentation. (hossman, Steve Rowe) + +================== 7.4.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.17 +Carrot2 3.16.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.11 +Jetty 9.4.10.v20180503 + +Upgrade Notes +---------------------- + +* SOLR-7887: Solr now uses Log4J 2.11 . The log4j configuration is now in log4j2.xml rather than log4j.properties files. + This is a server side change only and clients using SolrJ won't need any changes. Clients can still use any logging + implementation which is compatible with SLF4J. We now let log4j2 handle rotation of solr logs at startup, and bin/solr + start scripts will no longer attempt this nor move existing console or GC logs into logs/archived either (SOLR-12144). + +* SOLR-11673: Slave doesn't commit empty index when completely new index is detected on master during replication. + To return the previous behavior pass false to skipCommitOnMasterVersionZero in slave section of replication + handler configuration, or pass it to the fetchindex command. + +* SOLR-11453: Configuring slowQueryThresholdMillis now logs slow requests to a separate file - solr_slow_requests.log. + Previously they would get logged in the solr.log file. + +New Features +---------------------- + +* SOLR-12396: Upgrade Carrot2 to 3.16.0, HPPC to 0.8.1, morfologik to 2.1.5. (Dawid Weiss) + +* SOLR-11200: A new CMS config option 'ioThrottle' to manually enable/disable + ConcurrentMergeSchedule.doAutoIOThrottle. (Amrit Sarkar, Nawab Zada Asad iqbal via Dawid Weiss) + +* SOLR-11670: Implement a periodic house-keeping task. This uses a scheduled autoscaling trigger and + currently performs cleanup of old inactive shards. (ab, shalin) + +* SOLR-12015: Add support "add-distinct" in AtomicURP so that we can use the 'add-distict' as a request parameter e.g: + atomic.<multival-field-name>=add-distict (Amrit Sarkar via noble) + +* SOLR-7887: Upgrade Solr to use Log4J 2.11 + (Tim Potter, Keith Laban, Shawn Heisey, Ralph Goers, Erick Erickson, Varun Thacker) + +* SOLR-12139: The "eq" (equals) function query now works with string fields, string literals, and perhaps anything. + (Andrey Kudryavtsev, David Smiley) + +* SOLR-10783: Add support for Hadoop Credential Provider as SSL/TLS store password source. + (Mano Kovacs via Mark Miller) + +* SOLR-12036: Factor out DefaultStreamFactory solrj class. (Christine Poerschke) + +* SOLR-12151: Add abstract MultiSolrCloudTestCase class. (Christine Poerschke) + +* SOLR-12181: Add index size autoscaling trigger, based on document count or size in bytes. (ab) + +* SOLR-11982: Add possibility to define replica order with the shards.preference parameter to e.g. prefer PULL replicas + for distributed queries. (Ere Maijala, Tomás Fernández Löbbe) + +* SOLR-11336: DocBasedVersionConstraintsProcessorFactory is more extensible and now supports a list of versioned fields. + (versionField config may now be a comma-delimited list). (Michael Braun via David Smiley) + +* SOLR-11913: SolrJ SolrParams now implements Iterable<Map.Entry<String, String[]>> and also has a stream() method + using it for convenience. (David Smiley, Tapan Vaishnav) + +* SOLR-11924: Added the ability to listen to changes in the set of active collections in a cloud + in the ZkStateReader, through the CloudCollectionsListener. (Houston Putman, Dennis Gove) + +* SOLR-8998: introducing uniqueBlock(_root_) aggregation as faster alternative to unique(_root_) for counting + child value facets in parents via json.facet on block index (Dr Oleg Savrasov, Mikhail Khludnev) + +* SOLR-12278: Add IgnoreLargeDocumentProcessFactory (Cao Manh Dat, David Smiley) + +* SOLR-11277: Add auto hard-commit settings based on tlog size (Rupa Shankar, Anshum Gupta) + +* SOLR-9480: A new 'relatedness()' aggregate function for JSON Faceting to enable building Semantic + Knowledge Graphs. (Trey Grainger, hossman) + +* SOLR-11453: Configuring slowQueryThresholdMillis logs slow requests to a separate file - solr_slow_requests.log. + (Shawn Heisey, Remko Popma, Varun Thacker) + +* SOLR-12401: Add getValue() and setValue() Stream Evaluators (Joel Bernstein, janhoy) + +* SOLR-12378: Support missing versionField on indexed docs in DocBasedVersionConstraintsURP. + (Oliver Bates, Michael Braun via Mark Miller) + +* SOLR-12388: Enable a strict ZooKeeper-connected search request mode, in which search + requests will fail when the coordinating node can't communicate with ZooKeeper, + by setting the "shards.tolerant" param to "requireZkConnected". (Steve Rowe) + +* SOLR-9685: #Tagging queries in JSON Query DSL, equivalent to LocalParams based query/filter + tagging. Multiple tags are comma separated. + LocalParams Example : {!tag=colorfilt}color:blue + Equivalent JSON Example : { "#colorfilt" : "color:blue" } + (Dmitry Tikhonov, Mikhail Khludnev, yonik) + +* SOLR-12328: JSON Facet API: Domain change with graph query. + (Daniel Meehl, Kevin Watters, yonik) + +* SOLR-11453: Configuring slowQueryThresholdMillis logs slow requests to a separate file - solr_slow_requests.log. + (Shawn Heisey, Remko Popma, Varun Thacker) + +* SOLR-12401: Add getValue() and setValue() Stream Evaluators (Joel Bernstein, janhoy) + +* SOLR-11779, SOLR-12438: Basic long-term collection of aggregated metrics. Historical data is + maintained as multi-resolution time series using round-robin databases in the '.system' + collection. New /admin/metrics/history API allows retrieval of this data in numeric + or graph formats. (ab) + +* SOLR-12387: cluster-wide defaults for numShards, nrtReplicas, tlogReplicas, pullReplicas (noble) + +* SOLR-12389: support deeply nested json objects in clusterprops.json (noble) + +* SOLR-12376: Added the TaggerRequestHandler (AKA SolrTextTagger) for tagging text. It's used as a component of + NER/ERD systems including query-understanding. See the ref guide for more info. (David Smiley) + +* SOLR-12266: Add discrete Fourier transform Stream Evaluators (Joel Bernstein) + +* SOLR-12158: Allow the monteCarlo Stream Evaluator to support variables (Joel Bernstein) + +* SOLR-11734: Add ones and zeros Stream Evaluators (Joel Bernstein) + +* SOLR-12273: Create Stream Evaluators for distance measures (Joel Bernstein) + +* SOLR-12159: Add memset Stream Evaluator (Joel Bernstein) + +* SOLR-12221: Add valueAt Stream Evaluator (Joel Bernstein) + +* SOLR-12175: Add random field type and dynamic field to the default managed-schema (Joel Bernstein) + +Bug Fixes +---------------------- + +* SOLR-5351: Fixed More Like This Handler to use all fields provided in mlt.fl when used with + content stream. The similarity is calculated between the content stream's value and all + fields listed in mlt.fl. (Dawid Weiss) + +* SOLR-12103: Raise CryptoKeys.DEFAULT_KEYPAIR_LENGTH from 1024 to 2048. (Mark Miller) + +* SOLR-12107: Fixed a error in [child] transformer that could ocur if documentCache was not used (hossman) + +* SOLR-12108: Fixed the fallback behavior of [raw] and [xml] transformers when an incompatble 'wt' was + specified, the field value was lost if documentCache was not used. (hossman) + +* SOLR-11551: Standardize CoreAdmin API success/failure status codes (Jason Gerlowski, Steve Rowe) + +* SOLR-12035: ExtendedDismaxQParser fails to include charfilters in nostopanalyzer (Tim Allison via + Tomás Fernández Löbbe) + +* SOLR-10734: AtomicUpdateRequestProcessor can cause wrong/old values to be set under concurrent updates for the same + document. Multithreaded test for AtomicUpdateRequestProcessor was also beefed up and fixed. + (Ishan Chattopadhyaya, Noble Paul, Amrit Sarkar, shalin) + +* SOLR-11673: By default slave doesn't commit empty index when completely new index appears on master. + See Upgrade Notes to find a way to get back to the previous behavior. (Mikhail Khludnev) + +* SOLR-9399: Delete requests do not send credentials & fails for Basic Authentication + (Susheel Kumar, Aibao Luo, Nikkolay Martinov via Erick Erickson) + +* SOLR-12172: Fixed race condition that could cause an invalid set of collection properties to be kept in + memory when multiple collection property changes are done in a short period of time. (Tomás Fernández Löbbe) + +* SOLR-11929: UpdateLog metrics are not initialized on core reload. (ab, Steve Rowe) + +* SOLR-11882: SolrMetric registries retained references to SolrCores when closed. A + change of SolrMetricMAnager.registerGauge and SolrMetricProducer.initializeMetrics + method signatures was required to fix it. Third party components may continue to use the old API + but should be updated to avoid this bug (Eros Taborelli, Erick Erickson, ab) + +* SOLR-12199: TestReplicationHandler.doTestRepeater(): TEST_PORT interpolation failure: + Server refused connection at: http://127.0.0.1:TEST_PORT/solr (Mikhail Khludnev, Dawid Weiss, Steve Rowe) + +* SOLR-12096: Fixed inconsistent results format of subquery transformer for distributed search (multi-shard). + (Munendra S N, Mikhail Khludnev via Ishan Chattopadhyaya) + +* SOLR-12207: Just rethrowing AssertionError caused by jdk bug in reflection with invocation details. + (ab, Dawid Weiss, Mikhail Khludnev) + +* SOLR-12155: Exception from UnInvertedField constructor puts threads to infinite wait. + (Andrey Kudryavtsev, Mikhail Khludnev) + +* SOLR-12201: TestReplicationHandler.doTestIndexFetchOnMasterRestart(): handle unexpected replication failures + (Steve Rowe) + +* SOLR-12190: Need to properly escape output in GraphMLResponseWriter. (yonik) + +* SOLR-12214: Leader may skip publish itself as ACTIVE when its last published state is DOWN (Cao Manh Dat) + +* SOLR-12150: Fix a test bug in CdcrBidirectionalTest.testBiDir (Steve Rowe, Amrit Sarkar via Varun Thacker) + +* SOLR-10513: ConjunctionSolrSpellChecker did not work with LuceneLevenshteinDistance (Amrit Sarkar via James Dyer) + +* SOLR-11840: Fix bin/solr help-text inconsistencies (Jason Gerlowski) + +* SOLR-10169: PeerSync will hit an NPE on no response errors when looking for fingerprint. (Erick Erickson) + +* SOLR-12187: Replica should watch clusterstate and unload itself if its entry is removed (Cao Manh Dat) + +* SOLR-6286: TestReplicationHandler.doTestReplicateAfterCoreReload(): stop checking for identical + commits before/after master core reload; and make non-nightly mode test 10 docs instead of 0. + (shalin, hossman, Mark Miller, Steve Rowe) + +* SOLR-9304: Fix Solr's HTTP handling to respect '-Dsolr.ssl.checkPeerName=false' aka SOLR_SSL_CHECK_PEER_NAME + (Shawn Heisey, Carlton Findley, Robby Pond, hossman) + +* SOLR-12250: NegativeArraySizeException on TransactionLog if previous document more than 1.9GB (Cao Manh Dat) + +* SOLR-12253: Remove optimize button from the core admin page (Erick Erickson) + +* SOLR-11833: Allow searchRate trigger to delete replicas. Improve configurability of the trigger by specifying + upper / lower thresholds and respective actions (ab) + +* SOLR-12261: If you attempt to delete a collection immediately after deleting an alias it used to be a part of, you may + get an error that's it's a member of that alias. This check now ensures the alias state is sync()'ed with ZK first. + (David Smiley) + +* SOLR-12275: wrong caching for {!filters} as well as for `filters` local param in {!parent} and {!child} + (David Smiley, Mikhail Khluldnev) + +* SOLR-12284: WordBreakSolrSpellchecker will no longer add parenthesis in collations when breaking words in + non-boolean queries. (James Dyer) + +* SOLR-12290: Do not close any servlet streams and improve our servlet stream closing prevention code for users + and devs. (Mark Miller, janhoy, Andrzej Bialecki) + +* SOLR-12293: Updates need to use their own connection pool to maintain connection reuse and prevent spurious + recoveries. (Mark Miller) + +* SOLR-12308: LISTALIASES is now assured to return an up-to-date response. Also, the test utility + MiniSolrCloudCluster.deleteAllCollections will now first delete aliases since a collection cannot be deleted if an + alias refers to it. (David Smiley) + +* SOLR-12192: Fixed a solr cli error message when ulimits are unlimited. (Martijn Koster) + +* SOLR-12258: A V2 request referencing a collection or alias may fail to resolve it if it was just recently created. + Now we sync with ZooKeeper and try one more time. V1 partially did this but only for aliases; now it does both. + (David Smiley) + +* SOLR-12170: JSON Facet API: Terms facet on a date field sometimes failed with an exception complaining + about "Invalid Date String". (yonik) + +* SOLR-12307: exiting OverseerTriggerThread without endless noise in log when Zookeeper session is expired + (Mikhail Khludnev) + +* SOLR-12200: abandon OverseerExitThread when ZkController is closed. (Mikhail Khludnev) + +* SOLR-12355: Fixes hash conflict in HashJoinStream and OuterHashJoinStream (Dennis Gove) + +* SOLR-12377: Don't spin off overseer when Zk controller is closed (Mikhail Khludnev) + +* SOLR-3567: Spellcheck custom parameters not being passed through due to wrong prefix creation. + (Josh Lucas via shalin) + +* SOLR-12358: Autoscaling suggestions fail randomly with sorting (noble) + +* SOLR-12294: update processors loaded from runtime jars fail to load if they are specified + in an update processor chain (noble) + +* SOLR-12314: Use http timeout's defined in solr.xml for creating ConcurrentUpdateSolrClient during + indexing requests between leader and replica ( Mark Miller, Varun Thacker) + +* SOLR-12290: Do not close any servlet streams and improve our servlet stream closing prevention code for users + and devs. (Mark Miller) + +* SOLR-12374: SnapShooter.getIndexCommit can forget to decref the searcher; though it's not clear in practice when. + (David Smiley) + +* SOLR-12417: velocity response writer should enforce valid function name for v.json parameter (Mano Kovacs, yonik) + +* SOLR-12271: Fixed bug in how Analytics component reads negative values from float and double fields. (Houston Putman) + +* SOLR-12433: Recovering flag of a replica is set equals to leader even it failed to receive update + on recovering. (Cao Manh Dat) + +* SOLR-12354: Register the /admin/info/key end-point at the startup time to avoid 404 (noble) + +* SOLR-12445: Upgrade Dropwizard Metrics to version 3.2.6. (ab) + +* SOLR-12434: bin/solr {config,healthcheck} ignore ZK_HOST in solr.in.{sh,cmd} (Steve Rowe) + +* SOLR-12481: update.autoCreateFields must be set via Config API command 'set-user-property', + but 'bin/solr create' tells users to use the default action 'set-property', which fails + because the property is not editable. (Steve Rowe) + +* SOLR-12416: When creating a time routed alias, the router.autoDeleteAge option wasn't considered. + (Joachim Sauer via David Smiley) + +* SOLR-12450: Don't allow referal to external resources in various config files (CVE-2018-8026). + (Yuyang Xiao, Uwe Schindler) + +Optimizations +---------------------- + +* SOLR-11920: IndexFetcher now fetches only those files (from master/leader) that are different. This + differential fetching now speeds up recovery times when full index replication is needed, but only + a few segments diverge. (Ishan Chattopadhyaya, Shaun Sabo, John Gallagher) + +* SOLR-11731: LatLonPointSpatialField can now decode points from docValues when stored=false docValues=true, + albeit with maximum precision of 1.37cm (Karthik Ramachandran, David Smiley) + +* SOLR-11891: DocStreamer now respects the ReturnFields when populating a SolrDocument, reducing the + number of unneccessary fields a ResponseWriter will see if documentCache is used (wei wang, hossman) + +* SOLR-12312: Replication IndexFetcher should cap its internal buffer size when the file being transferred is small. + (Jeff Miller, David Smiley) + +* SOLR-12333: Removed redundant lines for handling lists in JSON reponse writers. (David Smiley via Mikhail Khludnev) + +* SOLR-11880: Avoid creating new exceptions for every request made to MDCAwareThreadPoolExecutor by distributed + search and update operations. (Varun Thacker, shalin) + +* SOLR-12375: Optimize Lucene ScoreMode use: + A non-cached filter query could be told incorrectly that scores were needed. + The /export (ExportQParserPlugin) would declare incorrectly that scores are needed. + Expanded docs (expand component) could be told incorrectly that scores are needed. (David Smiley) + +* SOLR-12338: Replay buffering tlog in parallel. (Cao Manh Dat, David Smiley) + +* SOLR-12366: A slow "live docs" implementation was being used instead of a bitset. Affects classic faceting + enum method, JSON Facets enum method, UnInvertedField faceting, GraphTermsQParser, JoinQParser. (David Smiley) + +* SOLR-12337: Remove the obsolete QueryWrapperFilter intermediate wrapper, which also removed some needless uses of + SolrConstantScoreQuery as well. QWF since v5.4.0 sometimes needlessly internally executed and cached the query. + Affects ExpandComponent, ChildDocTransformer, CurrencyFieldType, TermsQParser. (David Smiley) + +* SOLR-9922: Write buffering updates to another tlog. (Cao Manh Dat) + +* SOLR-12233: QParserPlugin's built-in static registry now holds actual QParserPlugin instances instead of class + references. This is consistent with other plugin registries and allows a SolrCore to load faster. + (Jeff Miller, David Smiley) + +* SOLR-12198: Stream Evaluators should not copy matrices needlessly (Joel Bernstein) + +Other Changes +---------------------- + +* SOLR-12018: Remove comments.apache.org integration for the Ref Guide; the comments system has been down since December 2017 and there is no concrete plan to bring it back. (Cassandra Targett) + +* SOLR-12076: Remove unnecessary printLayout usage in CDCR tests (Varun Thacker) + +* SOLR-12086: Fix format problem in FastLRUCache description string shown on Cache Statistics page. + (Sathiya N Sundararajan via shalin) + +* SOLR-12090: Move DistribStateManager, NodeStateProvider and SolrCloudManager interfaces out of the + autoscaling package. (shalin) + +* SOLR-12091: Rename TimeSource.getTime to getTimeNs. (ab) + +* SOLR-12101: ZkTestServer was not handling connection timeout settings properly. (Gus Heck via Mark Miller) + +* SOLR-11331: Ability to run and debug standalone Solr and a single node SolrCloud server from Eclipse. + Also being able to run all Lucene and Solr tests as a configuration (Karthik Ramachandran via + Varun Thacker, Uwe Schindler) + +* SOLR-12118: Solr Ref-Guide can now use some ivy version props directly as attributes in content (hossman) + +* SOLR-12152: Split up TriggerIntegrationTest into multiple tests to isolate and increase reliability. (shalin) + +* SOLR-12133: Fix race conditions that caused TriggerIntegrationTest.testEventQueue and testNodeMarkersRegistration + to fail. (Mark Miller, shalin) + +* SOLR-12169: Fix ComputePlanActionTest.testSelectedCollections fails on jenkins by aggressively cleaning up + trigger state left by other test methods in the test setup. (shalin) + +* SOLR-12144: SOLR_LOG_PRESTART_ROTATION now defaults to false, we leverage log4j2 for log rotation on startup (janhoy) + +* SOLR-12095: AutoScalingHandler validates trigger configurations before updating Zookeeper. (ab) + +* SOLR-12165: Ref Guide: DisMax default mm param value is improperly documented as 100%. (Steve Rowe) + +* SOLR-12154: Disallow explicit usage of Log4j2 logger via forbidden APIs. (Varun Thacker, Tomás Fernández Löbbe) + +* SOLR-12176: Improve FORCELEADER to handle the case when a replica win the election but does not present + in clusterstate (Cao Manh Dat) + +* SOLR-12134: ref-guide 'bare-bones html' validation is now part of 'ant documentation' and validates + javadoc links locally. (hossman) + +* SOLR-12142: EmbeddedSolrServer should use req.getContentWriter (noble) + +* SOLR-12252: Fix minor compiler and intellij warnings in autoscaling policy framework. (shalin) + +* SOLR-11914: The following SolrParams methods are now deprecated: toSolrParams (use NamedList.toSolrParams instead), + toMap, toMultiMap, toFilteredSolrParams, getAll. The latter ones have no direct replacement but are easy to + implement yourself as-needed. (David Smiley) + +* SOLR-10036: Upgrade jackson from 2.5.4 to 2.9.5 (Shashank Pedamallu, Shawn Heisey, Varun Thacker) + +* SOLR-12289: Add more MDC logging information to collection admin requests (Varun Thacker) + +* SOLR-12288: Add more MDC logging information to core admin requests (Varun Thacker) + +* SOLR-12265: Upgrade Jetty to 9.4.10.v20180503 (Varun Thacker, Steve Rowe) + +* SOLR-12374: Added SolrCore.withSearcher(lambda) to make grabbing the searcher easier than the more awkward + RefCounted API. (David Smiley) + +* SOLR-12183: Refactor Streaming Expression test cases (Joel Bernstein) + +* SOLR-12437: Document 'bin/solr config' in the ref guide. (Steve Rowe) + +* SOLR-12435: Fix bin/solr help and ref guide text to describe ZK_HOST in solr.in.sh/solr.in.cmd + as an alternative to -z cmdline param. (Steve Rowe) + +* SOLR-12428: Solr LTR jar now included in _default configset's solrconfig.xml (Ishan Chattopadhyaya) + +================== 7.3.1 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.17 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.11 +Jetty 9.4.8.v20171121 + +Bug Fixes +---------------------- + +* SOLR-12204: Upgrade commons-fileupload dependency to 1.3.3 to address CVE-2016-1000031. (Steve Rowe) + +* SOLR-12256: Fixed some eventual-consistency issues with collection aliases by using ZooKeeper.sync(). (David Smiley) + +* SOLR-12087: Deleting replicas sometimes fails and causes the replicas to exist in the down + state (Cao Manh Dat) + +* SOLR-12146: LIR should skip deleted replicas (Cao Manh Dat) + +* SOLR-12066: Cleanup deleted core when node start (Cao Manh Dat) + +* SOLR-12065: A successful restore collection should mark the shard state as active and not buffering + (Rohit, Varun Thacker) + +* SOLR-11724: Cdcr bootstrapping should ensure that non-leader replicas should sync with the leader + (Amrit Sarkar, Varun Thacker) + +* SOLR-12202: Fix errors in solr-exporter.cmd. (Minoru Osuka via koji) + +* SOLR-12316: Do not allow to use absolute URIs for including other files in solrconfig.xml + and schema parsing (CVE-2018-8010). (Ananthesh, Ishan Chattopadhyaya, Uwe Schindler) + +================== 7.3.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.16 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.11 +Jetty 9.4.8.v20171121 + +Upgrade Notes +---------------------- + +* SOLR-11748: The throttling mechanism used to limit the rate of autoscaling events processed + has been removed. This deprecates the 'actionThrottlePeriodSeconds' setting in the set-properties + Autoscaling API which is now a no-op. Use the 'triggerCooldownPeriodSeconds' instead to pause event + processing. + +* SOLR-11798: The top-level <highlighting> syntax in solrconfig.xml is now formally + deprecated in favour of <searchComponent> equivalent syntax. See also SOLR-1696. + +* SOLR-11809: QueryComponent's rq parameter parsing no longer considers the defType parameter. + +* SOLR-11747: The behaviour of the autoscaling system has been modified to pause all triggers from execution between + the start of actions and end of cool down period. The triggers will be resumed after the cool down period expires. + Previously, the cool down period was a fixed period started after actions for a trigger event complete and during + this time, all triggers continued to run but any events were rejected to be tried later. + +* SOLR-11624: Collections created without specifying a configset name use a copy of the _default configset since 7.0. + Before 7.3, the copied over configset was named the same as the collection name, but 7.3 onwards it will be named + with an additional ".AUTOCREATED" suffix. + +* SOLR-11702: The old Leader-In-Recovery implementation (SOLR-5495) is now deprecated and replaced. Solr will support + rolling upgrades from old 7.x versions of Solr to future 7.x releases until the last release of + the 7.x major version. This means in order to upgrade to Solr 8 in the future, you must be on Solr 7.3 or higher. + +* LUCENE-8161: If you are using the spatial JTS library with Solr, you must upgrade to 1.15.0. This new version + of JTS is now dual-licensed to include a BSD style license. + +* SOLR-12051: A new mechanism is introduced in SOLR-11702 to maintain consistency in SolrCloud between leader and replicas. + This mechanism lets Solr know whether a replica is in-sync with the leader or not, even when the leader is not live. + If all the replicas who participate in the leader election are out-of-sync with previous leader, the election will + pause until a timeout (named "leaderVoteWait") before allowing an out-of-sync replica to become leader. Note that the + new leader still needs to contains more updates than any other active replicas in the same shard. Therefore by + increasing leaderVoteWait will increase the consistency (over availability) of the system. The default value of + leaderVoteWait is 180,000 ms (3 minutes) and it can be adjusted in the "solrcloud" section of the solr.xml + +* SOLR-11957: The default Solr log file size and number of backups is raised to 32MB and 10 respectively + +* SOLR-12067: The default value of `autoReplicaFailoverWaitAfterExpiration` has been increased to 120 seconds + from the earlier default of 30 seconds. This affects how soon Solr adds new replicas to replace the replicas + on nodes which have either crashed or shutdown. + +New Features +---------------------- + +* SOLR-11285: Simulation framework for autoscaling. (ab) + +* LUCENE-2899: In the Solr analysis-extras contrib, added support for the + OpenNLP-based analysis components in the Lucene analysis/opennlp module: + tokenization, part-of-speech tagging, phrase chunking, and lemmatization. + Also added OpenNLP-based named entity extraction as a Solr update request + processor. (Lance Norskog, Grant Ingersoll, Joern Kottmann, Em, Kai Gülzau, + Rene Nederhand, Robert Muir, Steven Bower, Steve Rowe) + +* SOLR-11201: Implement autoscaling trigger for arbitrary metrics that creates events when + a given metric breaches a threshold (shalin) + +* SOLR-11653: TimeRoutedAlias URP now auto-creates new collections on the fly according to alias metadata + rules that sets the time interval for each collection. An internal Overseer command "ROUTEDALIAS_CREATECOLL" + was created to facilitate this. (David Smiley) + +* SOLR-11062: new tag "diskType" in autoscaling policy (noble) + +* SOLR-11063: Suggesters should accept required freedisk as a hint (noble) + +* SOLR-3218: Added range faceting support for CurrencyFieldType. This includes both "facet.range" as well + as json.facet's "type:range" (Andrew Morrison, Jan Høydahl, Vitaliy Zhovtyuk, hossman) + +* SOLR-11064: Collection APIs should use the disk space hint when using policy framework (noble) + +* SOLR-11854: multivalued primitive fields can now be sorted by implicitly choosing the min/max + value for asc/desc sort orders. (hossman) + +* SOLR-11592: Add OpenNLP language detection to the langid contrib. (Koji, Steve Rowe) + +* SOLR-11648: A new admin UI to display and execute suggestions (Apoorv Bhawsar , noble) + +* SOLR-11722: Added "time routed alias" creation support to the CREATEALIAS command. It's for managing multiple + collections partitioned by time. (Gus Heck, David Smiley) + +* SOLR-11782: Refactor LatchWatcher.await to protect against spurious wakeup (Tomás Fernández Löbbe, David Smiley, Dawid Weiss) + +* SOLR-11617: Alias properties (formerly "metadata") are now mutable via a new ALIASPROP command. + These properties are returned from LISTALIASES. (Gus Heck via David Smiley) + +* SOLR-11702: Redesign current LIR implementation (Cao Manh Dat, shalin) + +* SOLR-11376: Support computing plans for only specific collections. (ab) + +* SOLR-11681: Add ttest and pairedTtest Stream Evaluators (Joel Bernstein) + +* SOLR-11785: Add multiVariateNormalDistribution Stream Evaluator (Joel Bernstein) + +* SOLR-11789: Add integrate Stream Evaluator to support integral calculations (Joel Bernstein) + +* SOLR-11791: Add density Stream Evaluator (Joel Bernstein) + +* SOLR-11172: Add Mann-Whitney U test Stream Evaluator (Mathew Skaria, Joel Bernstein) + +* SOLR-11808: Add sumSq Stream Evaluator (Joel Bernstein) + +* SOLR-11430: Add lerp and akima Stream Evaluators to support linear and akima spline interpolation (Joel Bernstein) + +* SOLR-11683: Add chiSquareDataSet Stream Evaluator (Joel Bernstein) + +* SOLR-10716: Add termVectors Stream Evaluator (Joel Bernstein) + +* SOLR-11737: Add kmeans Stream Evaluator to support kmeans clustering (Joel Bernstein) + +* SOLR-11736: Rename knn Streaming Expression to knnSearch and add new knn Stream Evaluator (Joel Bernstein) + +* SOLR-11867: Add indexOf, rowCount and columnCount StreamEvaluators (Joel Bernstein) + +* SOLR-11862: Add fuzzyKmeans Stream Evaluatory (Joel Bernstein) + +* SOLR-11890: Add multiKmeans Stream Evaluator (Joel Bernstein) + +* SOLR-11916: new SortableTextField which supports analysis/searching just like TextField, but also + sorting/faceting just like StrField. By default uses only the first 1024 chars of the original + input string values, but this is configurable. (hossman) + +* SOLR-11778: Add per-stage RequestHandler metrics. (ab) + +* SOLR-11925: Time Routed Aliases can have their oldest collections automatically deleted via the "router.autoDeleteAge" + setting. (David Smiley) + +* SOLR-11941: Add abstract contrib/ltr AdapterModel to facilitate the development of scoring models that delegate + scoring to an opaque pre-trained model. (Christine Poerschke) + +* SOLR-11689: Add l1norm, l2norm and linfnorm Stream Evaluators (Joel Bernstein) + +* SOLR-11588: Add matrixMult Stream Evaluator to support matrix multiplication (Joel Bernstein) + +* SOLR-12006: Add a '*_t' and '*_t_sort' dynamic field for single valued text fields (Varun Thacker) + +* SOLR-11597: Add contrib/ltr NeuralNetworkModel class. (Michael A. Alcorn, Yuki Yano, Christine Poerschke) + +* SOLR-11066: Implement a scheduled autoscaling trigger that runs on a fixed interval beginning with a + given start time. (David Smiley, Gus Heck, ab, shalin) + +* SOLR-11795: Add Solr metrics exporter for Prometheus (Minoru Osuka via koji) + +* SOLR-11267: Add support for "add-distinct" atomic update operation (Amrit Sarkar via noble ) + +* SOLR-11960: Add collection level properties similar to cluster properties (Peter Rusko, Tomás Fernández Löbbe) + +* SOLR-12077: Add support for autoAddReplicas in the collection creation dialog in Admin UI. (shalin) + +* SOLR-9510: introducing {!filters param=$fq excludeTags=f} query parser. + Introducing {!.. filters=$fq excludeTags=t,q} in {!parent} and {!child} (Dr. Oleg Savrasov via Mikhail Khludnev) + +Bug Fixes +---------------------- + +* SOLR-11783: Rename core in solr standalone mode is not persisted (Erick Erickson) + +* SOLR-11555: If the query terms reduce to nothing, filter(clause) produces an NPE whereas + fq=clause does not (Erick Erickson) + +* SOLR-11824: Fixed bucket ordering in distributed json.facet type:range when mincount>0 (hossman) + +* SOLR-11821: ConcurrentModificationException in SimSolrCloudTestCase.tearDown (shalin) + +* SOLR-11631: The Schema API should return non-zero status when there are failures. + (Noble Paul, Steve Rowe) + +* SOLR-11839: Fix test failures resulting from SOLR-11218 (Erick Erickson) + +* SOLR-11794: PULL replicas stop replicating after collection RELOAD (Samuel Tatipamula, Tomás Fernández Löbbe) + +* SOLR-11714: AddReplicaSuggester / ComputePlanAction infinite loop. (ab) + +* SOLR-11895: Logging Screen in the Admin UI will now show "No Events available" when there are no events to show. + Also, the spinner icon is removed to prevent users thinking the page has hung. (Cassandra Targett, Shawn Heisey) + +* SOLR-10525: Stacked recovery requests do no cancel an in progress recovery first. (Mike Drob via Cao Manh Dat) + +* SOLR-11873: Use time based expiration cache in all necessary places in HdfsDirectoryFactory. (Mihaly Toth via Mark Miller) + +* SOLR-11661: New HDFS collection reuses unremoved data from a deleted HDFS collection with same name causes + inconsistent view of documents (Cao Manh Dat, shalin) + +* SOLR-11459: In-place update of nonexistent doc following existing doc update fails to create the doc. + (Andrey Kudryavtsev via Mikhail Khludnev) + +* SOLR-11931: Fix contrib/ltr custom inner class feature/normaliser/model persistence. (Christine Poerschke) + +* SOLR-10261: In case of in-place updates, failure in leader to follower replica update request now throws the + follower replica in leader-initiated-recovery (Ishan Chattopadhyaya, Steve Rowe) + +* SOLR-11898: ConcurrentModificationException when calling org.apache.solr.core.SolrInfoBean.getMetricsSnapshot + (Jeff Miller via Erick Erickson) + +* SOLR-11950: Allow CLUSTERSTATUS "shard" parameter to accept comma (,) delimited list (Chris Ulicny via Jason Gerlowski) + +* SOLR-11739: Fix race condition that made Solr accept duplicate async IDs in collection API operations (Tomás Fernánadez Löbbe) + +* SOLR-11988: Fix exists() method in EphemeralDirectoryFactory/MockDirectoryFactory to prevent false positives (hossman) + +* SOLR-11971: Don't allow referal to external resources in DataImportHandler's dataConfig request parameter (CVE-2018-1308). + (麦 香浓郁, Uwe Schindler) + +* SOLR-12021: Fixed a bug in ApiSpec and other JSON resource loading that was causing unclosed file handles (hossman) + +* SOLR-10720: Aggressive removal of a collection breaks cluster status API. (Alexey Serba, shalin) + +* SOLR-12050: UTILIZENODE does not enforce policy rules (hossman, noble) + +* SOLR-11843: Admin UI collection creation was not properly handling the router.field and router.name parameters. + Also changed the display label in the Admin UI from routerField to router.field to match the actual API. + (Shawn Heisey via Cassandra Targett) + +* SOLR-12011: Consistence problem when in-sync replicas are DOWN. (Cao Manh Dat) + +* SOLR-12020: JSON Facet API: terms facet on date field fails in refinement phase with + "Invalid Date String" error. (yonik) + +* SOLR-12061: Fix substitution bug in API V1 to V2 migration when using SolrJ with V2 API. (Tomás Fernánadez Löbbe) + +* SOLR-11976: TokenizerChain.normalize: only the first filter that is a MultiTermAwareComponent was participating + in normalization instead of all. This bug normally doesn't matter since TextField doesn't call it. + (Tim Allison via David Smiley) + +* SOLR-12072: Invalid path string using ZkConfigManager.copyConfigDir(String fromConfig, String toConfig) + (Alessandro Hoss via Erick Erickson) + +* SOLR-12064: JSON Facet API: fix bug where a limit of -1 in conjunction with multiple facets or + missing=true caused an NPE or AIOOBE. (Karthik Ramachandran via yonik) + +* SOLR-12083: Fix RealTime GET to work on a cluster running CDCR when using Solr's in-place updates + (Amrit Sarkar, Varun Thacker) + +* SOLR-12063: Fix PeerSync, Leader Election failures and CDCR checkpoint inconsistencies on a cluster running CDCR + (Amrit Sarkar, Varun Thacker) + +* SOLR-12110: Replica which failed to register in Zk can become leader (Cao Manh Dat) + +* SOLR-12129: After the core is reloaded, term of the core will not be watched (Cao Manh Dat) + +* SOLR-12141: Fix "bin/solr" shell scripts (Windows/Linux/Mac) to correctly detect major Java version + and use numerical version comparison to enforce minimum requirements. Also remove obsolete "UseParNewGC" option. + This allows to start Solr with Java 10 or later. (Uwe Schindler) + +Optimizations +---------------------- + +* SOLR-11711: Fixed distributed processing of facet.field/facet.pivot sub requests to prevent requesting + unneccessary and excessive '0' count terms from each shard (Houston Putman via hossman) + +* LUCENE-8149: NRTCachingDirectory does not need to preemptively delete segment files and generate exceptions + (Erick Erickson) + +* SOLR-11879: avoid EOFException for empty input streams (noble) + +* SOLR-8327: Cluster state caching to avoid live fetch from ZK of cluster state for forwarding requests on nodes + that do not host any replica of the collection (Jessica Cheng Mallet, John Gallagher, Noble Paul, Ishan Chattopadhyaya) + +* SOLR-11769: Optimize when useFilterForSortedQuery=true and there are no filter queries by avoiding + needless match-all-docs bitset construction. (Betim Deva, David Smiley) + +Other Changes +---------------------- + +* SOLR-11629: Add more intuitive CloudSolrClient.Builder constructors (Varun Thacker, Jason Gerlowski) + +* SOLR-11575: Improve ref-guide solrj snippets via mock 'print()' method (Jason Gerlowski via hossman) + +* SOLR-11757: In tests, fix race condition on SolrException.ignoreException. + Also ensure we register "ignore_exception" in @BeforeClass (previously only @AfterClass) (David Smiley) + +* SOLR-11754: Remove AbstractSolrTestCase which has long been supplanted by SolrTestCaseJ4. (David Smiley) + +* SOLR-11701: Upgrade to Tika 1.17 when available (Tim Allison, Karthik Ramachandran via Erick Erickson) + +* SOLR-11703: Solr Should Send Log Notifications if Ulimits are too low (Kevin Cowan via Erick Eickson) + +* SOLR-11793: Reduce code duplication w.r.t. RestTestHarness(es). (Christine Poerschke) + +* SOLR-7733: remove "optimize" from the UI. (Erick Erickson) + +* SOLR-11748: Remove Autoscaling action throttle. (shalin) + +* SOLR-11805: SolrJ's SolrResponse.getElaspedTime was sometimes a millisecond off. (David Smiley) + +* SOLR-11798: Formally deprecate top-level <highlighting> syntax in solrconfig.xml + in favour of <searchComponent> equivalent syntax. (Christine Poerschke) + +* SOLR-11801: Support customisation of the "highlighting" query response element. + (Ramsey Haddad, Pranav Murugappan, David Smiley, Christine Poerschke) + +* SOLR-11692: SolrDispatchFilter's use of a "close shield" in tests should not be applied to + further servlet chain processing. (Jeff Miller, David Smiley) + +* SOLR-11218: Fail and return an error when attempting to delete a collection that's part of an alias (Erick Erickson) + +* SOLR-11817: Move Collections API classes to it's own package (Varun Thacker) + +* SOLR-11810: Upgrade Jetty to 9.4.8.v20171121 (Varun Thacker, Erick Erickson) + +* SOLR-11747: Pause triggers until actions finish executing and the cool down period expires. (shalin) + +* SOLR-11871: MoveReplicaSuggester should not suggest leader if other replicas are available (noble) + +* SOLR-11624: Collections created from _default configset will now be associated with a configset with a suffix + .AUTOCREATED. For example, a new collection "mycollection", created without specifying a configset name, will + use the _default configset and the associated configset name will be "mycollection.AUTOCREATED". If this + collection is deleted and re-created, the autocreated configset will be left behind and will be re-used for + the re-created collection (Ishan Chattopadhyaya, Abhishek Kumar Singh) + +* SOLR-11051: Use disk free metric in default cluster preferences (noble) + +* SOLR-11658: Upgrade ZooKeeper dependency to 3.4.11 (Jason Gerlowski, Steve Rowe, Erick Erickson) + +* SOLR-11480: Remove unused "Admin Extra" files and mentions. (Eric Pugh, Christine Poerschke) + +* SOLR-11067: REPLACENODE should identify appropriate nodes if targetNode is not provided (noble) + +* SOLR-11848: Update Ref Guide to include info on grouping operations and using curl for large files. (Dariusz Wojtas + via Cassandra Targett) + +* SOLR-11613: Make message for missing dataimport config in UI more explicit. (Shawn Heisey, Amrit Sarkar via + Cassandra Targett) + +* SOLR-11933: Make DIH UI page safer by not default checking the clean checkbox (Eric Pugh via Tomás Fernández Löbbe) + +* SOLR-11349: Rename ResponseBuilder's getQueryCommand to createQueryCommand. (Christine Poerschke) + +* SOLR-11902: Clarify in bin/solr help text whether commands can be run remotely (Jason Gerlowski) + +* SOLR-3089: RequestBuilder now exposes isDistrib() method. Using this, plugins can know whether a request is + distributed. (Rok Rejc, Frank Wesemann, Abhishek Kumar Singh, Mikhail Khludnev via Ishan Chattopadhyaya) + +* SOLR-8090: Make text elements in the UI darker for better contrast and readability (Cassandra Targett) + +* SOLR-6057: Change highlight color in UI Analysis screen (Cassandra Targett) + +* SOLR-11897: Add toggle to disable auto-refresh of Logging page in the UI (Tommy Marshment-Howell via + Cassandra Targett) + +* SOLR-12017: Remove BadApple and AwaitsFix annotations that link to closed JIRAs (Erick Erickson) + +* SOLR-12027: Increase thread lingering timeout to 80s. (Mikhail Khludnev) + +* SOLR-10809: Get precommit lint warnings out of Solr core (Erick Erickson) + +* SOLR-12028: BadApple and AwaitsFix annotations usage (Erick Erickson, Uwe Schindler) + +* SOLR-12031: Refactor Policy framework to make simulated changes affect more than a single node (noble) + +* SOLR-11957: Increase MaxFileSize=32MB and MaxBackupIndex=10 for RollingFileAppender in log4j.properties + (Varun Thacker, shalin) + +* SOLR-12047: Increase checkStateInZk timeout (Cao Manh Dat, Varun Thacker) + +* SOLR-12051: Election timeout when no replicas are qualified to become leader (Cao Manh Dat) + +* SOLR-12067: Increase autoAddReplicas default 30 second wait time to 120 seconds. + (Varun Thacker, Mark Miller via shalin) + +* SOLR-12078: Fixed reproducable Failure in TestReplicationHandler.doTestIndexFetchOnMasterRestart that happened + due to using stale http connections. (Gus Heck, shalin) + +* SOLR-12099: Remove reopenReaders attribute from 'IndexConfig in SolrConfig' page in ref guide. (shalin) + +* SOLR-12098: Document the Lucene spins auto-detection and its effect on CMS dynamic defaults. + (Cassandra Targett, shalin) + +* SOLR-12097: Document the diskType policy attribute and usage of disk space in Collection APIs. + (Cassandra Targett, shalin) + +================== 7.2.1 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.16 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.10 +Jetty 9.3.20.v20170531 + +Bug Fixes +---------------------- + +* SOLR-11771: Overseer can never process some last messages (Cao Manh Dat) + +* SOLR-11783: Rename core in solr standalone mode is not persisted (Erick Erickson) + +* SOLR-11809: QueryComponent.prepare rq parsing could fail under SOLR 7.2.0 - fix: + QueryComponent's rq parameter parsing no longer considers the defType parameter. + (Christine Poerschke and David Smiley in response to bug report/analysis + from Dariusz Wojtas and Diego Ceccarelli) + +* SOLR-11555: If the query terms reduce to nothing, filter(clause) produces an NPE whereas + fq=clause does not (Erick Erickson) + +================== 7.2.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.16 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.10 +Jetty 9.3.20.v20170531 + +Upgrade Notes +---------------------- + +* SOLR-11501: Starting a query string with local-params {!myparser ...} is used to switch the query parser to another, + and is intended for use by Solr system developers, not end users doing searches. To reduce negative side-effects of + unintended hack-ability, we've limited the cases that local-params will be parsed to only contexts in which the + default parser is "lucene" or "func". + So if defType=edismax then q={!myparser ...} won't work. In that example, put the desired query parser into defType. + Another example is if deftype=edismax then hl.q={!myparser ...} won't work for the same reason. In that example, + either put the desired query parser into hl.qparser or set hl.qparser=lucene. Most users won't run into these cases + but some will and must change. + If you must have full backwards compatibility, use luceneMatchVersion=7.1.0 or something earlier. (David Smiley) + +* SOLR-11501: The edismax parser by default no longer allows subqueries that specify a Solr parser using either + local-params, or the older _query_ magic field trick. For example + {!prefix f=myfield v=enterp} or _query_:"{!prefix f=myfield v=enterp}" are not supported by default anymore. + If you want to allow power-users to do this, set uf=* _query_ or some other value that includes _query_. + If you must have full backwards compatibility, use luceneMatchVersion=7.1.0 or something earlier. (David Smiley) + +New Features +---------------------- +* SOLR-11448: Implement an option in collection commands to wait for final results. (ab) + +* SOLR-11072: Implement trigger for searchRate event type. (ab) + +* SOLR-11524: A new autoscaling/suggestions API end-point which gives autoscaling suggestions (noble) + +* SOLR-11519: Implement autoscaling suggestions for replica count violations (noble) + +* SOLR-11518: Implement autoscaling Suggestions for freedisk violations (noble) + +* SOLR-10132: A new optional facet.matches parameter to return facet buckets only + for terms that match a regular expression. (Gus Heck, Christine Poerschke) + +* SOLR-11520: Implement autoscaling suggestions for cores count violations (noble) + +* SOLR-11438: Solr should return rf when min_rf is specified for deletes as well as adds (Erick Erickson) + +* SOLR-11003: Support bi-directional syncing of cdcr clusters. We still only support active indexing in one cluster, + but have the ability to switch indexing clusters and cdcr will replicate correctly. (Amrit Sarkar, Varun Thacker) + +* SOLR-11538: Implement suggestions for port,ip_*, nodeRole,sysprop.*, metrics:* (noble) + +* SOLR-11487: Collection Aliases may now have metadata (currently an internal feature). + (Gus Heck, David Smiley) + +* SOLR-11542: New TimeRoutedAliasUpdateProcessor URP that routes documents to another collection + in the same Alias defined set based on a time field (currently an internal feature). + (David Smiley) + +* SOLR-9743: A new UTILIZENODE command (noble) + +* SOLR-11202: Implement a set-property command for AutoScaling API. (ab, shalin) + +* SOLR-11250: A new DefaultWrapperModel class for loading of large and/or externally stored + LTRScoringModel definitions. (Yuki Yano, shalin, Christine Poerschke) + +* SOLR-11662: New synonymQueryStyle option to configure whether SynonymQuery, a DisjunctionMaxQuery, or BooleanQuery + occurs over query terms that overlap their position. (Doug Turnbull, David Smiley) + +* SOLR-11429: Add loess Stream Evaluator to support local regression (Joel Bernstein) + +* SOLR-11568: Add matrix Stream Evaluator to support efficient matrix operations (Joel Bernstein) + +* SOLR-11566: Add transpose Stream Evaluator to support transposing of matrices (Joel Bernstein) + +* SOLR-11565: Add unit Stream Evaluator to support unitizing of vectors (Joel Bernstein) + +* SOLR-11567: Add triangularDistribution Stream Evaluator (Joel Bernstein) + +* SOLR-11593: Add support for covariance matrices to the cov Stream Evaluator (Joel Bernstein) + +* SOLR-10680: Add minMaxScale Stream Evaluator (Joel Bernstein) + +* SOLR-11599: Change normalize function to standardize and make it work with matrices (Joel Bernstein) + +* SOLR-11602: Add Markov Chain Stream Evaluator (Joel Bernstein) + +* SOLR-11607: Add grandSum, sumRows, sumColumns, scalarDivide, scalarMultiply, scalarAdd, + scalarSubtract Stream Evaluators (Joel Bernstein) + +* SOLR-11571: Add diff Stream Evaluator to support time series differencing (Mathew Skaria, Joel Bernstein) + +* SOLR-11570: Add support for correlation matrices to the corr Stream Evaluator (Joel Bernstein) + +* SOLR-11569: Add support for distance matrices to the distance Stream Evaluator (Joel Bernstein) + +* SOLR-11674: Support ranges in the probability Stream Evaluator (Joel Bernstein) + +* SOLR-11680: Add normalizeSum Stream Evaluator (Joel Bernstein) + +* SOLR-11697: Add geometricDistribution Stream Evaluator (Joel Bernstein) + +* SOLR-11485: Add olsRegress, spline and derivative Stream Evaluators (Joel Bernstein) + +Bug Fixes +---------------------- + +* SOLR-11423: Overseer queue needs a hard cap (maximum size) that clients respect (Scott Blum, Joshua Humphries, Noble Paul) + +* SOLR-11445: Overseer should not hang when process bad message. (Cao Manh Dat, shalin) + +* SOLR-11447: ZkStateWriter should process commands in atomic. (Cao Manh Dat, shalin) + +* SOLR-11491: Support retrieval of cluster properties in HttpClusterStateProvider. (ab) + +* SOLR-11146: Small fixes to the mapping functions in the Analytics Component (Houston Putman) + +* SOLR-11326: A bootstrap of a target cluster does not need to download the tlog files from the source cluster + (Amrit Sarkar, Varun Thacker) + +* SOLR-10874: Fix AIOOBE caused by FloatPayloadValueSource calling PostingsEnum.nextPosition() + more than the given term's frequency in overridden FloatDocValues.floatVal(). + (Michael Kosten, Erik Hatcher, Steve Rowe) + +* SOLR-11484: CloudSolrClient does not invalidate cache or retry for RouteException (noble, hossman) + +* SOLR-11532: Solr hits NPE when fl only contains DV fields and any of them is a spatial field. (Cao Manh Dat, David Smiley) + +* SOLR-11557: Fix SolrZkClient.checkInterrupted (Tomás Fernández Löbbe) + +* SOLR-11413: SolrGraphiteReporter fails to report metrics due to non-thread safe code. (Erik Persson, ab) + +* SOLR-9440: The ZkStateReader.removeCollectionStateWatcher method can cache a DocCollection reference and + never update it causing stale state to be returned in ClusterState. (shalin) + +* SOLR-11560: Specifying the replicationFactor parameter while restoring a collection would lead to extra tlog + and pull replicas being created (Peter Szantai-Kis, Varun Thacker) + +* SOLR-11586: Restored collection should use stateFormat=2 instead of 1. (Varun Thacker) + +* SOLR-11503: Collections created with legacyCloud=true cannot be opened if legacyCloud=false (Erick Erickson) + +* SOLR-11619: V2 requests that needed to be forwarded to other nodes would get an NPE. (David Smiley) + +* SOLR-11231: Guard against unset fields when performing language detection. + (Chris Beer via Steve Rowe) + +* SOLR-11553: JSON Facet API: facet refinement could use UIF in phase one and DV uninversion in phase 2, resulting + in too much memory use. (yonik) + +* SOLR-11645: When the java commandline had exact duplicate arguments, the + admin UI dashboard would not display any commandline arguments. + (Webster Homer via Shawn Heisey) + +* SOLR-11608: Correctly parse the new core-name in the V2 core rename API. + (Jason Gerlowski via Anshum Gupta) + +* SOLR-11256: The queue size for ConcurrentUpdateSolrClient should default to 10 instead of throwing an + IllegalArgumentException. (Jason Gerlowski, Anshum Gupta) + +* SOLR-11616: Snapshot the segments more robustly such that segments created during a backup does does not fail the + operation (Varun Thacker) + +* SOLR-11687: SolrCore.getNewIndexDir falsely returns {dataDir}/index on any IOException reading index.properties + (Nikolay Martynov, Erick Erickson) + +* SOLR-9137: bin/solr script ignored custom STOP_PORT on shutdown. + (Joachim Kohlhammer, Steve Rowe, Christine Poerschke) + +* SOLR-11458: Improve error handling in MoveReplicaCmd to avoid potential loss of data. (ab) + +* SOLR-11590: Synchronize ZK connect/disconnect handling so that they are processed in linear order + (noble, Varun Thacker) + +* SOLR-11664: JSON Facet API: range facets containing unique, hll, min, max aggregations over string fields + produced incorrect results since 7.0 (Volodymyr Rudniev, yonik) + +* SOLR-11691: V2 requests for create-alias didn't work when the collections param was an array. + (Jason Gerlowski, Gus Heck, David Smiley, noble) + +Optimizations +---------------------- +* SOLR-11285: Refactor autoscaling framework to avoid direct references to Zookeeper and Solr + to make testing and large scale simulations easier. (ab) + +* SOLR-11443: Remove the usage of workqueue for Overseer. (Cao Manh Dat, Scott Blum) + +* SOLR-11320: Lock autoscaling triggers when changes they requested are being made. This helps to + ensure that cluster is in a stable state before processing any new trigger events. (ab) + +* SOLR-11641: Change `frange` to default to `cost=100` so default behavior is to PostFilter if user specifies + `cache=false` (hossman) + +* SOLR-11595: SolrIndexSearcher.localCollectionStatistics is now faster by leveraging an existing + cached MultiFields. Noticeable when many fields are searched. (David Smiley) + +Other Changes +---------------------- +* SOLR-11478: Solr should remove itself from live_nodes in zk immediately on shutdown. (Cao Manh Dat) + +* SOLR-11145: Adds comprehensive unit tests for the Analytics Component v2 (Houston Putman) + +* SOLR-11389: For Solr(Shard|Cluster)Reporter instances the SolrMetricManager.registerReporter + method is now called after the SolrCore or CoreContainer has been set for the instance. + (Christine Poerschke) + +* SOLR-11464, SOLR-11493, SOLR-11511: Minor refactorings to DistributedUpdateProcessor. (Gus Heck, David Smiley) + +* SOLR-11444: Improved consistency of collection alias handling, and collection references that are + comma delimited lists of collections, across the various places collections can be referred to. + Updates are delivered to the first in a list. It's now possible to refer to a comma delimited list + of them in the path, ex: /solr/colA,colB/select?... (David Smiley) + +* SOLR-11165: Ref guide documentation for Autoscaling APIs, triggers, listeners, actions for Solr 7.1 + (ab, Cassandra Targett, Noble Paul, shalin) + +* SOLR-11562: Restore Solr logo ASCII-art in startup log by removing unnecessary default confdir logging (janhoy) + +* SOLR-11380: SolrJ must stream docs to server instead of writing to a buffer first (noble) + +* SOLR-11603: Remove unused (public) LTRScoringModel.hasParams() method. (Christine Poerschke) + +* SOLR-11606: Disable tests automatically if Mockito does not work with Java runtime (Java 10). + (Uwe Schindler) + +* SOLR-11618: Tone down verbosity of BackupManager logging (Varun Thacker) + +* SOLR-11621: Fix spurious failures of TriggerIntegrationTest due to timing issues. (shalin) + +* SOLR-11628: Add documentation of maxRamMB for filter cache and query result cache. (shalin) + +* SOLR-11610: Refactored payload handling to use lucene's PayloadDecoder + framework (Alan Woodward) + +* SOLR-9120: Reduce log level for inconsequential NoSuchFileException that LukeRequestHandler may encounter (hossman) + +* SOLR-10469: Move CloudSolrClient.setParallelUpdates to its Builder. (David Smiley) + +* SOLR-11507: SOLR-11638: Randomize SolrTestCaseJ4.CloudSolrClientBuilder more, and simplify it. + (Jason Gerlowski, David Smiley) + +* SOLR-11291: Factor out abstract metrics/SolrCore[Container]Reporter classes. + (Omar Abdelnabi, Christine Poerschke) + +* SOLR-11713: Fixed CdcrUpdateLogTest.testSubReader() failure which was a test bug (Varun Thacker, Amrit Sarkar) + +================== 7.1.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.16 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.10 +Jetty 9.3.20.v20170531 + +Detailed Change List +---------------------- + +Upgrade Notes +---------------------- + +* 'autoAddReplicas' feature is ported to autoscaling framework. Existing users of this feature should not have + to change anything. Note these changes: + 1. Behaviour: Changing the autoAddReplicas property from disabled to enabled using MODIFYCOLLECTION API + no longer replaces down replicas for the collection immediately. Instead, replicas are only added + if a node containing them went down while autoAddReplicas was enabled. The params + autoReplicaFailoverBadNodeExpiration and autoReplicaFailoverWorkLoopDelay are no longer used. + 2. Deprecations: Enabling/disabling autoAddReplicas cluster wide with the API will be deprecated; use + suspend/resume trigger APIs with name='.auto_add_replicas' instead. + +* SOLR-11195: shard and cluster metric reporter configuration now requires a class attribute. + If a reporter configures the group="shard" attribute then please also configure the + class="org.apache.solr.metrics.reporters.solr.SolrShardReporter" attribute. + If a reporter configures the group="cluster" attribute then please also configure the + class="org.apache.solr.metrics.reporters.solr.SolrClusterReporter" attribute. + +* SOLR-11254: the abstract DocTransformer class now has an abstract score-less transform method variant. + +* SOLR-11283: all Stream Evaluators in solrj.io.eval have been refactored to have a simplier and more + robust structure. This simplifies and condenses the code required to implement a new Evaluator and + makes it much easier for evaluators to handle differing data types (primitives, objects, arrays, + lists, and so forth). (Dennis Gove) + +* SOLR-10962: in the ReplicationHandler the master.commitReserveDuration sub-element is deprecated. Instead + please configure a direct commitReserveDuration element for use in all modes (master, slave, cloud). + +* SOLR-11482: RunExecutableListener was removed for security reasons. If you want to listen to + events caused by updates, commits, or optimize, write your own listener as native Java class + as part of a Solr plugin. + +* SOLR-11477: in the XML query parser (defType=xmlparser or {!xmlparser ... }) + the resolving of external entities is now disallowed by default. + +New Features +---------------------- +* SOLR-10339: New set-trigger and remove-trigger APIs for autoscaling. (shalin) + +* SOLR-10340: New set-listener and remove-listener API for autoscaling. (shalin) + +* SOLR-10358: New suspend-trigger and resume-trigger APIs for autoscaling. (shalin) + +* SOLR-10376: Implement autoscaling trigger for nodeAdded event. (shalin) + +* SOLR-10396: Implement trigger support for nodeLost event type (Cao Manh Dat, shalin) + +* SOLR-10496: New ComputePlanAction for autoscaling which uses the policy framework to compute cluster + operations upon a trigger fire. (Noble Paul, shalin) + +* SOLR-10965: New ExecutePlanAction for autoscaling which executes the operations computed by ComputePlanAction + against the cluster. (shalin) + +* SOLR-11019: Add addAll Stream Evaluator (Joel Bernstein) + +* SOLR-10996: Implement TriggerListener API (ab, shalin) + +* SOLR-11046: Add residuals Stream Evaluator (Joel Bernstein) + +* SOLR-10858: Make UUIDUpdateProcessorFactory as Runtime URP (Amit Sarkar, noble) + +* SOLR-11031: Implement SystemLogListener for autoscaling (ab) + +* SOLR-11205: Any metrics value can be directly accessed in autoscaling policies (noble) + +* SOLR-11199: Payloads supports an "operator" param. Supported operators are 'or', "phrase" ( default ). + A new "sum" function is also added. Example : + {!payload_score f=payload_field func=sum operator=or}A B C" (Varun Thacker) + +* SOLR-11215: Make a metric accessible through a single param. (ab) + +* SOLR-11244: Query DSL for Solr (Cao Manh Dat) + +* SOLR-11317: JSON Facet API: min/max aggregations on numeric fields are now typed better so int/long + fields return an appropriate integral type rather than a double. (yonik) + +* SOLR-11316: JSON Facet API: min/max aggregations are now supported on single-valued date fields. + (yonik) + +* SOLR-10962: Make ReplicationHandler's commitReserveDuration configurable in SolrCloud mode. + (Ramsey Haddad, Christine Poerschke, hossman) + +* SOLR-11382: Lucene's Geo3D (surface of sphere & ellipsoid) is now supported on spatial RPT fields by + setting spatialContextFactory="Geo3D". Furthermore, this is the first time Solr has out of the box + support for polygons. (David Smiley) + +* SOLR-11160: Add normalDistribution, uniformDistribution, sample and kolmogorovSmirnov Stream Evaluators + (Joel Bernstein) + +* SOLR-11225: Add cumulativeProbability Stream Evaluator (Joel Bernstein) + +* SOLR-11241: Add discrete counting and probability Stream Evaluators (Joel Bernstein) + +* SOLR-11321: Add ebeAdd, ebeSubtract, ebeDivide, ebeMultiply, dotProduct and cosineSimilarity Stream Evaluators + (Joel Bernstein) + +* SOLR-11338: Add Kendall's Tau-b rank and Spearmans rank correlation Stream Evaluators (Joel Bernstein) + +* SOLR-11339: Add Canberra, Chebyshev, Earth Movers and Manhattan Distance Stream Evaluators (Joel Bernstein) + +* SOLR-11342: Add sumDifference and meanDifference Stream Evaluators (Joel Bernstein) + +* SOLR-11350: Add primes Stream Evaluator (Joel Bernstein) + +* SOLR-11354: Add factorial and movingMedian Stream Evaluators (Joel Bernstein) + +* SOLR-11377: Add expMovingAverage (exponential moving average) and binomialCoefficient Stream Evaluators + (Mathew Skaria, Joel Bernstein) + +* SOLR-11388: Add monteCarlo Stream Evaluator to support Monte Carlo simulations (Joel Bernstein) + +* SOLR-11398: Add weibullDistribution Stream Evaluator (Joel Bernstein) + +* SOLR-11400: Add logNormalDistribution Stream Evaluator (Joel Bernstein) + +* SOLR-11401: Add zipFDistribution Stream Evaluator (Joel Bernstein) + +* SOLR-11414: Add gammaDistribution Stream Evaluator (Joel Bernstein) + +* SOLR-11415: Add betaDistribution Stream Evaluator (Joel Bernstein) + +* SOLR-11436: Add polyfit and polyfitDerivative Stream Evaluators (Joel Bernstein) + +* SOLR-11439: Add harmonicFit Stream Evaluator (Joel Bernstein) + +* SOLR-11076: New /autoscaling/history API to return past cluster events and actions. (ab) + +Bug Fixes +---------------------- + +* SOLR-10602: Triggers should be able to restore state from old instances when taking over. (shalin) + +* SOLR-10714: OverseerTriggerThread does not start triggers on overseer start until autoscaling + config watcher is fired. (shalin) + +* SOLR-10738: TriggerAction is initialised even if the trigger is never scheduled. (shalin) + +* SOLR-10668: fix NPE at sort=childfield(..) .. on absent values (Mikhail Khludnev) + +* SOLR-8984: EnumField's error reporting to now indicate the field name in failure log (Lanny Ripple, + Ann Addicks via Ishan Chattopadhyaya) + +* SOLR-11012: Fix three (JavaBinCodec not being closed) Resource Leak warnings. (Christine Poerschke) + +* SOLR-11026: MoveReplicaSuggester must check if the target becomes more 'loaded' than the source + if an operation is performed (noble) + +* SOLR-10994: CREATE, CREATESHARD & ADDREPLICA to use policy framework & support replica types (noble) + +* SOLR-11011: Assign.buildCoreName can lead to error in creating a new core when legacyCloud=false (Cao Manh Dat) + +* SOLR-10944: Get expression fails to return EOF tuple (Susheel Kumar, Joel Bernstein) + +* SOLR-6086: Replica is active during autowarming resulting in queries being sent to a replica that + may not have a registered searcher. This causes spikes in response times when adding a replica + in busy clusters. (Ludovic Boutros, Timothy Potter, shalin) + +* SOLR-11190: GraphQuery also supports string fields which are indexed=false and docValues=true. Please refer to the + Javadocs for DocValuesTermsQuery for it's performance characteristics. (Karthik Ramachandran, Varun Thacker) + +* SOLR-11084: Issue with starting script with solr.home (-s) == solr (Leil Ireson, Amrit Sarkar via Erick Erickson) + +* SOLR-11255: Fix occasional ConcurrentModificationException when using SolrInfoMBeanHandler. (ab) + +* SOLR-11272: fix NPE when EmbeddedSolrServer handles /admin/* request and so one (Stephen Allen via Mikhail Khludnev) + +* SOLR-11289: fix comma handling in terms component. (yonik) + +* SOLR-11164, SOLR-11180, SOLR-11220: Fix NullPointerException and always-returns-zero + contrib/ltr OriginalScoreFeature issues in SolrCloud mode. + (Yuki Yano, Jonathan Gonzalez, Ryan Yacyshyn, Christine Poerschke) + +* SOLR-11278: Stopping CDCR should cancel a running bootstrap operation. (Amrit Sarkar, shalin) + +* SOLR-11293: Potential data loss in TLOG replicas when masterVersion equals zero (noble, Cao Manh Dat) + +* SOLR-10101: TestLazyCores hangs (Erick Erickson) + +* SOLR-11332: Fix sorting on 'enum' fieldTypes that use sortMissingFirst or sortMissingLast (hossman) + +* SOLR-11348: Fix the DIH database example (James Dyer) + +* SOLR-11363: JSON Facet API: repeated values in a numeric points field with docValues enabled + were double counted. (Hossman, yonik) + +* SOLR-11085: Improve resiliency of autoscaling actions against overseer restarts and operation failures. (shalin) + +* SOLR-11297: Message "Lock held by this virtual machine" during startup. Solr is trying to start some cores twice. + (Luiz Armesto, Shawn Heisey, Erick Erickson) + +* SOLR-11399: The UnifiedHighlighter was ignoring the hl.fragsize parameter when hl.bs.type=SEPARATOR + (Marc Morissette via David Smiley) + +* SOLR-11224: SolrStream.close can hit an NPE (Erick Erickson) + +* SOLR-11278: Fix a race condition in the CDCR bootstrap process which could lead to bootstraps cancelling itself + (Amrit Sarkar, shalin, Varun Thacker) + +* SOLR-11425: SolrClientBuilder does not allow infinite timeout (value 0). (Peter Szantai-Kis via Mark Miller) + +* SOLR-11449: MoveReplicaCmd mistakenly called registerCollectionStateWatcher on failure. (ab) + +* SOLR-11477: Disallow resolving of external entities in the XML query parser (defType=xmlparser). + (Michael Stepankin, Olga Barinova, Uwe Schindler, Christine Poerschke) + +* SOLR-12444: Updating a cluster policy fails (noble) + +Optimizations +---------------------- + +* SOLR-10985: Remove unnecessary toString() calls in solr-core's search package's debug logging. + (Michael Braun via Christine Poerschke) + +* SOLR-11000: Changes made via AutoScalingHandler should be atomic. (ab, shalin) + +* SOLR-11124: MoveReplicaCmd should skip deleting old replica in case of its node is not live (Cao Manh Dat) + +* SOLR-10769: Allow nodeAdded / nodeLost events to report multiple nodes in one event. (ab) + +* SOLR-11242: QueryParser: re-use the LookaheadSuccess exception. (Michael Braun via David Smiley) + +* SOLR-11314: FastCharStream: re-use the READ_PAST_EOF exception. (Michael Braun via David Smiley) + +* SOLR-8344: Decide default when requested fields are both column and row stored. (Cao Manh Dat, David Smiley) + +* SOLR-10285: Skip LEADER messages when there are leader only shards (Cao Manh Dat, Joshua Humphries) + +* SOLR-11424: When legacyCloud=false, cores should not publish itself as DOWN on startup. (Cao Manh Dat) + +Other Changes +---------------------- +* SOLR-10643: Throttling strategy for triggers and policy executions. (shalin) + +* SOLR-10764: AutoScalingHandler should validate policy and preferences before updating zookeeper. (shalin) + +* SOLR-10827: Factor out abstract FilteringSolrMetricReporter class. (Christine Poerschke) + +* SOLR-10957: Changed SolrCoreParser.init to use the resource loader from getSchema() + instead of the resource loader from getCore(). (Christine Poerschke) + +* SOLR-10748: Make stream.body configurable and disabled by default (janhoy) + +* SOLR-10964: Reduce SolrIndexSearcher casting in LTRRescorer. (Christine Poerschke) + +* SOLR-11075: Refactor handling of params in CloudSolrStream and FacetStream (Erick Erickson) + +* SOLR-11052: Remove unnecessary Long-to-Integer and back casts in ReplicationHandler. + (Ramsey Haddad via Christine Poerschke) + +* SOLR-11106: TestLBHttpSolrClient.testReliablity takes 30 seconds because of the wrong server name + (Kensho Hirasawa via Erick Erickson) + +* SOLR-10338: Configure SecureRandom non blocking for tests. (Mihaly Toth, hossman, Ishan Chattopadhyaya, via Mark Miller) + +* SOLR-10916: Convert tests that extend LuceneTestCase and use MiniSolrCloudCluster + to instead extend SolrCloudTestCase. (Steve Rowe) + +* SOLR-11131: Document 'assert' as a command option in bin/solr, and bin/solr.cmd scripts. + (Jason Gerlowski via Anshum Gupta) + +* SOLR-11140: Remove unused parameter in (private) SolrMetricManager.prepareCloudPlugins method. + (Omar Abdelnabi via Christine Poerschke) + +* SOLR-11187: contrib/ltr TestModelManagerPersistence improvements. (Yuki Yano via Christine Poerschke) + +* SOLR-10397: Port 'autoAddReplicas' feature to the autoscaling framework and make it work with non-shared filesystems + (Cao Manh Dat, shalin) + +* SOLR-11157: remove-policy must fail if a policy to be deleted is used by a collection (noble) + +* SOLR-11090: Add Replica.getProperty accessor. (Christine Poerschke) + +* SOLR-11061: Add a spins metric for data directory paths. (ab) + +* SOLR-11071: Improve TestIntervalFacets.testRandom (Tomás Fernández Löbbe) + +* SOLR-11195: Require class attribute for shard and cluster metric reporter configuration. (Christine Poerschke) + +* SOLR:10822: Share a Policy.session object between multiple collection admin calls to get the correct computations (noble) + +* SOLR-11249: Upgrade Jetty from 9.3.14.v20161028 to 9.3.20.v20170531 (Michael Braun via David Smiley) + +* SOLR-11270: Automatically added SystemLogListener should not prevent deleting a trigger. (ab) + +* SOLR-5129: Timeout property for waiting ZK get started. (Cao Manh Dat, Hrishikesh Gadre, Varun Thacker) + +* SOLR-10628: Less verbose output from bin/solr commands. (Jason Gerlowski, janhoy) + +* SOLR-11240: Raise UnInvertedField internal limit. (Toke Eskildsen) + +* SOLR-11254: Add score-less (abstract) DocTransformer.transform method. (Christine Poerschke) + +* SOLR-11209: Upgrade HttpClient to 4.5.3. (Hrishikesh Gadre via Mark Miller) + +* SOLR-11322: JSON Facet API: instead of returning NaN, min & max aggregations omit + the value for any bucket with no values in the numeric field. (yonik) + +* SOLR-10990: Breakup QueryComponent.process method for readability. (Christine Poerschke) + +* SOLR-11132: Refactor common getSortField logic in various FieldTypes (Jason Gerlowski, hossman) + +* SOLR-11351: Make LTRScoringModel model more extensible. (Christine Poerschke) + +* SOLR-10451: Remove contrib/ltr/lib from lib includes in the techproducts example config (sungjunyoung via janhoy) + +* SOLR-11306: Fix inaccurate comments on docValues and StrField in the example schemas + (Tom Burton-West, Jason Gerlowski, Varun Thacker) + +* SOLR-11482: RunExecutableListener was removed for security reasons. (Michael Stepankin, + Olga Barinova, Uwe Schindler, Tomás Fernández Löbbe) + +* SOLR-10335: Upgrade to Tika 1.16. (Tim Allison, Tomás Fernández Löbbe via shalin) + +================== 7.0.1 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.10 +Jetty 9.3.14.v20161028 + +Bug Fixes +---------------------- + +* SOLR-11297: Message "Lock held by this virtual machine" during startup. Solr is trying to start some cores twice. + (Luiz Armesto, Shawn Heisey, Erick Erickson) + +* SOLR-11406: Solr 7.0 cannot read indexes from 6.x versions. (Shawn Heisey, Steve Rowe) + +================== 7.0.0 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.10 +Jetty 9.3.14.v20161028 + +Upgrading from Solr 6.x +---------------------- + +* All Trie* numeric and date field types have been deprecated in favor of *Point field types. + Point field types are better at range queries (speed, memory, disk), however simple field:value queries underperform + relative to Trie. Either accept this, or continue to use Trie fields. + This shortcoming may be addressed in a future release. + +* The default response type is now JSON ("wt=json") instead of XML, and line indentation is now on by default + ("indent=on"). If you expect the responses to your queries to be returned in the previous format (XML + format, no indentation), you must now you must now explicitly pass in "wt=xml" and "indent=off" as query + parameters, or configure them as defaults on your request handlers. See SOLR-10494 for more details. + +* the cluster property 'legacyCloud' is set to false from 7.0. This means 'zookeeper is the truth' by + default. If an entry for a replica does not exist in the state.json, that replica cannot get + registered. This may affect users who use that feature where they bring up replicas and they are + automatically registered as a part of a shard. However, it is possible to fall back to the old behavior by + setting the property legacyCloud=true , in the cluster properties using the following command + + ./server/scripts/cloud-scripts/zkcli.sh -zkhost 127.0.0.1:2181 -cmd clusterprop -name legacyCloud -val true + +* HttpClientInterceptorPlugin is now HttpClientBuilderPlugin and must work with a + SolrHttpClientBuilder rather than an HttpClientConfigurer. + +* HttpClientUtil now allows configuring HttpClient instances via SolrHttpClientBuilder + rather than an HttpClientConfigurer. Use of env variable SOLR_AUTHENTICATION_CLIENT_CONFIGURER + no longer works, please use SOLR_AUTHENTICATION_CLIENT_BUILDER + +* SolrClient implementations now use their own internal configuration for socket timeouts, + connect timeouts, and allowing redirects rather than what is set as the default when + building the HttpClient instance. Use the appropriate setters on the SolrClient instance. + +* HttpSolrClient#setAllowCompression has been removed and compression must be enabled as + a constructor param. + +* HttpSolrClient#setDefaultMaxConnectionsPerHost and + HttpSolrClient#setMaxTotalConnections have been removed. These now default very + high and can only be changed via param when creating an HttpClient instance. + +* Query time join with scoring {!join score=none} (even none) doesn't handle single value numeric fields. + Users are advised to convert these fields into string and reindex. + +* Deprecated collection and configset methods on MiniSolrCloudCluster have been + removed + +* Index-time boosts are not supported anymore. If any boosts are provided, they + will be ignored by the indexing chain. As a replacement, index-time scoring + factors should be indexed in a separate field and combined with the query + score using a function query. + +* Deprecated method getNumericType() has been removed from FieldType. Use getNumberType() instead + +* MBean names and attributes now follow hierarchical names used in metrics. This is reflected also in + /admin/mbeans and /admin/plugins output, and can be observed in the UI Plugins tab, because now all these + APIs get their data from the metrics API. The old (mostly flat) JMX view has been removed. + +* <jmx> element in solrconfig.xml is no longer supported. Equivalent functionality can be configured in + solr.xml using <metrics><reporter ...> element and SolrJmxReporter implementation. Limited back-compatibility + is offered by automatically adding a default instance of SolrJmxReporter if it's missing, AND when a local + MBean server is found (which can be activated either via ENABLE_REMOTE_JMX_OPTS in solr.in.sh or via system + properties, eg. -Dcom.sun.management.jmxremote). This default instance exports all Solr metrics from all + registries as hierarchical MBeans. This behavior can be also disabled by specifying a SolrJmxReporter + configuration with a boolean init arg "enabled" set to "false". For a more fine-grained control users + should explicitly specify at least one SolrJmxReporter configuration. + +* The sow (split-on-whitespace) request param now defaults to false (true in previous versions). + This affects the edismax and standard/"lucene" query parsers: if the sow param is not specified, + query text will not be split on whitespace before analysis. See + https://lucidworks.com/2017/04/18/multi-word-synonyms-solr-adds-query-time-support/ . + +* Setting <solrQueryParser defaultOperator="..."/> in schema is no longer allowed and will cause an exception. + Please use "q.op" parameter on the request instead. For more details, see SOLR-10584. + +* Setting <defaultSearchField> in schema is no longer allowed and will cause an exception. + Please use "df" parameter on the request instead. For more details, see SOLR-10585. + +* The PostingsSolrHighlighter is deprecated. Furthermore, it now internally works via a re-configuration + of the UnifiedSolrHighlighter. + +* Deprecated LatLonType, GeoHashField, SpatialPointVectorFieldType, and SpatialTermQueryPrefixTreeFieldType. + Instead, switch to LatLonPointSpatialField or SpatialRecursivePrefixTreeFieldType or RptWithGeometrySpatialField. + +* The 'lucenePlusSort' QParser has been deprecated, and is no longer implicitly defined. If you wish to continue using + this QParser untill Solr 8, you must explicitly register it in your solrconfig.xml: + '<queryParser name="lucenePlusSort" class="solr.OldLuceneQParserPlugin"/>' + +* In solrconfig.xml the deprecated <mergePolicy> and <mergeFactor> and <maxMergeDocs> elements have + been removed in favor of the <mergePolicyFactory> element introduced by SOLR-8621 in Solr 5.5.0. + +* Custom SolrMetricReporter classes must implement a new, factored out doInit() method and the validate() method + must pass for all reporters including reporters that are not enabled, please see SOLR-10671 for details. + +* TemplateUpdateRequestProcessorFactory uses {} instead of ${} for template + +* SOLR-9565: The name of TemplateUpdateRequestProcessorFactory' is changed to 'template' from 'Template' and the + name of 'AtomicUpdateProcessorFactory' is changed to 'atomic' from 'Atomic' + +* The default for eDismax parameter 'lowercaseOperators' now depends on the luceneMatchVersion setting in solrconfig. + It remains 'true' for luceneMatchVersion < 7.0.0 and changes to 'false' from 7.0.0. See also SOLR-4646 + +* The unused 'valType' option has been removed from ExternalFileField, if you have this in your schema you + can safely remove it. see SOLR-10929 for more details. + +* Config sets basic_configs and data_driven_schema_configs have now been merged into _default. It has data driven nature + enabled by default, and can be turned off (after creating a collection) with: + curl http://host:8983/solr/mycollection/config -d '{"set-user-property": {"update.autoCreateFields":"false"}}' + Please see SOLR-10574 for details. + +* The data driven config (now _default) for auto-creating fields earlier defaulted to "string" for text input. + Now the default is to use "text_general" and to add a copyField to the schema, copying to a "*_str" dynamic field, + with a cutoff at 256 characters. This enables full text search as well as faceting. See SOLR-9526 for more. + +* SOLR-10123: The Analytics Component has been upgraded to support distributed collections, expressions over multivalued + fields, a new JSON request language, and more. DocValues are now required for any field used in the analytics expression + whereas previously docValues was not required. Please see SOLR-10123 for details. + +* solrconfig.xml: <requestDispatcher handleSelect="..."> now defaults to false when luceneMatchVersion >= 7.0, + thus ignoring "qt". Regardless, "qt" is still used as a SolrJ special parameter that specifies the request handler + (tail URL path) to use. If you have request handlers without a leading '/', you can set handleSelect="true" + or better yet consider migrating. + +* StandardRequestHandler is deprecated. Simply use SearchHandler instead. + +* The parameter names 'fromNode' for MOVEREPLICA and 'source', 'target' for REPLACENODE have been deprecated and + replaced with 'sourceNode' and 'targetNode' instead. The old names will continue to work for back-compatibility + but they will be removed in 8.0. See SOLR-11068 for more details. + +* All deperated methods of ClusterState (except getZkClusterStateVersion()) + have been removed. Use DocCollection methods instead. + +* SOLR-11023: EnumField has been deprecated in favor of new EnumFieldType. + +* SOLR-11239: The use of maxShardsPerNode is not supported when a cluster policy is in effect or + when a collection specific policy is specified during collection creation. + +* V2 APIs are now available at /api, in addition to /v2 (which is now deprecated). Legacy APIs continue to remain + available at /solr. + +* Solr was tested and is compatible with the final release candidate of Java 9. All startup scripts + detect Java 9 correctly and setup Garbage Collector logging. If the configuration file contains + logging options that are no longer supported with Java 9, startup will fail. + +* SOLR-10307: If starting Jetty without the Solr start script, you must now pass keystore and truststore + passwords via the env variables SOLR_SSL_KEY_STORE_PASSWORD and SOLR_SSL_TRUST_STORE_PASSWORD rather + than system properties. + +* SOLR-10379: ManagedSynonymFilterFactory has been deprecated in favor of ManagedSynonymGraphFilterFactory. + +* SOLR-10503: CurrencyField has been deprecated in favor of new CurrencyFieldType. + +New Features +---------------------- +* SOLR-9857, SOLR-9858: Collect aggregated metrics from nodes and shard leaders in overseer. (ab) + +* SOLR-9835: Create another replication mode for SolrCloud + +* SOLR-10356: Adds basic math Streaming Evaluators (Dennis Gove) + +* SOLR-10393: Adds UUID Streaming Evaluator (Dennis Gove) + +* SOLR-10046: Add UninvertDocValuesMergePolicyFactory class. (Keith Laban, Christine Poerschke) + +* SOLR-10547: JSON Facet API: Implement support for single-valued string fields for min/max aggregations. + (yonik) + +* SOLR-10262: Add support for configurable metrics implementations. (ab) + +* SOLR-10431: Make it possible to invoke v2 api calls using SolrJ (Cao Manh Dat, Noble Paul, shalin) + +* SOLR-10233: Add support for different replica types, that can handle updates differently: + - NRT: Writes updates to transaction log and indexes locally. Replicas of type “NRT” support NRT + (soft commits) and RTG. Any NRT replica can become a leader. This was the only type supported + in SolrCloud until now and it’s the default type. + - TLOG: Writes to transaction log, but not to index, uses replication to copy segment files from the + shard leader. Any TLOG replica can become leader (by first applying all local transaction log + elements). If a replica is of type TLOG but is also the leader, it will behave as a NRT. This + is exactly what was added in SOLR-9835 (non-realtime replicas), just the API and naming changes. + - PULL: Doesn’t index or writes to transaction log, just replicates from the shard leader. PULL replicas + can’t become shard leaders (i.e., if there are only PULL replicas in the collection at some point, + updates will fail same as if there is no leaders, queries continue to work), so they don’t even + participate in elections. + (Tomás Fernández Löbbe) + +* SOLR-10278: A new DSL to set cluster-wide preferences and policies on how to allocate replicas to nodes + (noble, shalin) + +* SOLR-10373: Implement read API for autoscaling configuration at /admin/autoscaling or + /cluster/autoscaling paths. (shalin) + +* SOLR-10677: Expose a diagnostics API to return nodes sorted by load in descending order and + any policy violations. (shalin) + +* SOLR-7452: Refinement for JSON Facet API: Adding refine:true to any terms facet will cause an + additional distributed search phase (overlapped with field retrieval) that requests additional info + for top facet buckets from shards that did not previously contribute to that bucket. + This will correct counts (and other statistics) for those top buckets collected in the first + phase. (yonik) + +* SOLR-10433: CollectionAdmin requests in SolrJ to support V2 calls (noble) + +* SOLR-9989: Add support for PointFields in FacetModule (JSON Facets) (Cao Manh Dat) + +* SOLR-10406: v2 API error messages list the URL request path as /solr/____v2/... when the original path was /v2/... (Cao Manh Dat, noble) + +* SOLR-10574: New _default config set replacing basic_configs and data_driven_schema_configs. + (Ishan Chattopadhyaya, noble, shalin, hossman, David Smiley, janhoy, Alexandre Rafalovitch) + +* SOLR-10272: Use _default config set if no collection.configName is specified with CREATE (Ishan Chattopadhyaya) + +* SOLR-9526: Data driven schema now indexes text field "foo" as both "foo" (text_general) and as "foo_str" (string) + to facilitate both search and faceting. AddSchemaFieldsUpdateProcessor now has the ability to add a "copyField" to + the type mappings, with an optional maxChars limitation. You can also define one typeMappings as default. + This also solves issues SOLR-8495, SOLR-6966, and SOLR-7058 + (janhoy, Steve Rowe, hossman, Alexandre Rafalovitch, Shawn Heisey, Cao Manh Dat) + +* SOLR-10123: Upgraded the Analytics Component to version 2.0 which now supports distributed collections, expressions over + multivalued fields, a new JSON request language, and more. DocValues are now required for any field used in the analytics + expression whereas previously docValues was not required. Please see SOLR-10123 for details. (Houston Putman) + +* SOLR-10282: bin/solr support for enabling Kerberos authentication (Ishan Chattopadhyaya, Jason Gerlowski) + +* SOLR-11093: Add support for PointFields for {!graph} query. (yonik) + +* SOLR-10845: Add support for PointFields to {!graphTerms} query that is internally + used by some graph traversal streaming expressions. (yonik) + +* SOLR-10939: Add support for PointsFields to {!join} query. Joined fields should + also have docValues enabled. (yonik) + +* SOLR-11173 TermsComponent support for Points fields. (yonik) + +* SOLR-10849: MoreLikeThisComponent should expose setMaxDocFreqPct (maxDoc + frequency percentage). (Dawid Weiss) + +* SOLR-10307: Allow Passing SSL passwords through environment variables. (Mano Kovacs, Michael Suzuki via Mark Miller) + +* SOLR-10379: Add ManagedSynonymGraphFilterFactory, deprecate ManagedSynonymFilterFactory. (Steve Rowe) + +* SOLR-10479: Adds support for HttpShardHandlerFactory.loadBalancerRequests(MinimumAbsolute|MaximumFraction) + configuration. (Ramsey Haddad, Daniel Collins, Christine Poerschke) + +* SOLR-3702: concat(...) function query (Andrey Kudryavtsev via Mikhail Khludnev) + +* SOLR-10767: Add movingAvg Stream Evaluator (Joel Bernstein) + +* SOLR-10813: Add arraySort Stream Evaluator (Joel Bernstein) + +* SOLR-10696: Add cumulative probability function (Joel Bernstein) + +* SOLR-10765: Add anova Stream Evaluator (Joel Bernstein) + +* SOLR-10754: Add hist Stream Evaluator (Joel Bernstein) + +* SOLR-10753: Add array Stream Evaluator (Joel Bernstein) + +* SOLR-10747: Allow /stream handler to execute Stream Evaluators directly (Joel Bernstein) + +* SOLR-10743: Add sequence StreamEvaluator (Joel Bernstein) + +* SOLR-10684: Add finddelay Stream Evaluator (Joel Bernstein) + +* SOLR-10731: Add knn Streaming Expression (Joel Bernstein) + +* SOLR-10724: Add describe Stream Evaluator (Joel Bernstein) + +* SOLR-10693: Add copyOfRange Stream Evaluator (Joel Bernstein) + +* SOLR-10623: Add sql Streaming Expression (Joel Bernstein) + +* SOLR-10661: Add copyOf Stream Evaluator (Joel Bernstein) + +* SOLR-10663: Add distance Stream Evaluator (Joel Bernstein) + +* SOLR-10664: Add scale Stream Evaluator (Joel Bernstein) + +* SOLR-10666: Add rank transformation Stream Evaluator (Joel Bernstein) + +* SOLR-10662: Add length Stream Evaluator (Joel Bernstein) + +* SOLR-10660: Add reverse Stream Evaluator (Joel Bernstein) + +* SOLR-9910: Add solr/solr.cmd parameter to append jetty parameters to the start script. + (Mano Kovacs via Mark Miller) + +* SOLR-6671: Possible to set solr.data.home property as root dir for all data (janhoy, Shawn Heisey, Mark Miller) + +Bug Fixes +---------------------- +* SOLR-9262: Connection and read timeouts are being ignored by UpdateShardHandler after SOLR-4509. + (Mark Miller, shalin) + +* SOLR-9837: Fix 55% performance regression of FieldCache uninvert time of + numeric fields. (yonik) + +* SOLR-10408: v2 API introspect should return useful message for non-existent command (Cao Manh Dat) + +* SOLR-10411: v2 Collection API "modify" command specification's replicationFactor property is incorrectly typed as string, + should be integer (Cao Manh Dat) + +* SOLR-10405: v2 API introspect response contains multiple copies of the experimental format WARNING (Cao Manh Dat) + +* SOLR-10412: v2 API: many API command specification properties are typed "number" but should instead be typed "integer" + (Steve Rowe, Cao Manh Dat) + +* SOLR-10413: v2 API: parsed JSON type should be coerced to expected type (Cao Manh Dat, Noble Paul) + +* SOLR-10223: Allow running examples as root on Linux with -force option (janhoy) + +* SOLR-10830: Solr now correctly enforces that the '_root_' field has the same fieldType as the + uniqueKey field. With out this enforcement, child document updating was unreliable. (hossman) + +* SOLR-10876: Regression in loading runtime UpdateRequestProcessors like TemplateUpdateProcessorFactory (noble) + +* SOLR-10886: Using V2Request.process(solrClient) method throws NPE if the API returns an error (Cao Manh Dat) + +* SOLR-10921: Work around the static Lucene BooleanQuery.maxClauseCount that causes Solr's maxBooleanClauses + setting behavior to be "last core wins". This patch sets BooleanQuery.maxClauseCount to its maximum value, + thus disabling the global check, and replaces it with specific checks where desired via + QueryUtils.build(). (yonik) + +* SOLR-10948: Fix extraction component to treat DatePointField the same as TrieDateField (hossman) + +* SOLR-10506: Fix memory leak (upon collection reload or ZooKeeper session expiry) in ZkIndexSchemaReader. + (Torsten Bøgh Köster, Christine Poerschke, Jörg Rathlev, Mike Drob) + +* SOLR-6807: CloudSolrClient's ZK state version check with the server was ignored when handleSelect=false + (David Smiley) + +* SOLR-10878: MOVEREPLICA command may lose data when replicationFactor is 1. (ab, shalin) + +* SOLR-10879: DELETEREPLICA and DELETENODE commands should prevent data loss when + replicationFactor is 1. (ab) + +* SOLR-10983: Fix DOWNNODE -> queue-work explosion (Scott Blum, Joshua Humphries) + +* SOLR-10826: Fix CloudSolrClient to expand the collection parameter correctly (Tim Owen via Varun Thacker) + +* SOLR-11039: Next button in Solr admin UI for collection list pagination does not work. (janhoy) + +* SOLR-11041: MoveReplicaCmd do not specify ulog dir in case of HDFS (Cao Manh Dat) + +* SOLR-11045: The new replica created by MoveReplica will have to have same name and coreName as the + old one in case of HDFS (Cao Manh Dat) + +* SOLR-11043: Fix facet.range.method=dv and interval facets on single-valued float fields with negative values. + (Tomás Fernández Löbbe, Steve Rowe) + +* SOLR-11073: Fix overflow in interval faceting when querying Long limits (e.g. (Long.MAX_VALUE TO Long.MAX_VALUE]) + (Tomás Fernández Löbbe) + +* SOLR-11057: Fix overflow in point range queries when querying the type limits + (e.g. q=field_i:{Integer.MAX_VALUE TO Integer.MAX_VALUE]) (Tomás Fernández Löbbe) + +* SOLR-11136: Fix solrj XMLResponseParser when nested docs transformer is used with indented XML (hossman) + +* SOLR-11130: V2Request in SolrJ should return the correct collection name so that the request is forwarded to the + correct node (noble) + +* SOLR-11151: SolrInfoMBeanHandler.getDiff() ADD case non-functional: NPE when a bean value goes from null -> non-null. + (Steve Rowe) + +* SOLR-11154: Child documents' return fields now include useDocValuesAsStored fields (Mohammed Sheeri Shaketi Nauage via + Ishan Chattopadhyaya) + +* SOLR-11163: Fix contrib/ltr Normalizer persistence after solr core reload or restart. + (Yuki Yano via Christine Poerschke) + +* SOLR-11182: A split shard failure on IOException should be logged (Varun Thacker) + +* SOLR-10353: TestSQLHandler reproducible failure: No match found for function signature min(<NUMERIC>) (Kevin Risden) + +* SOLR-11221: SolrJmxReporter broken on core reload. This resulted in some or most metrics not being reported + via JMX after core reloads, depending on timing. (ab) + +* SOLR-11235: Some SolrCore metrics should check if core is closed before reporting. (ab) + +* SOLR-10698: StreamHandler should allow connections to be closed early (Joel Bernstein, Varun Thacker, Erick Erickson) + +* SOLR-11243: Replica Placement rules are ignored if a cluster policy exists. (shalin) + +* SOLR-11268: AtomicUpdateProcessor complains missing UpdateLog (noble, Ishan Chattopadhyaya) + +* SOLR-8689: Fix bin/solr.cmd so it can run properly on Java 9 (Uwe Schindler, hossman) + +* SOLR-10723 JSON Facet API: resize() implemented incorrectly for CountSlotAcc, HllAgg.NumericAcc + resulting in exceptions when using a hashing faceting method and sorting by hll(numeric_field). + (yonik) + +* SOLR-10719: Creating a core.properties fails if the parent of core.properties is a symlinked dierctory + (Erick Erickson) + +* SOLR-10360: Solr HDFS snapshot export fails due to FileNotFoundException error when using MR1 instead of + yarn. (Hrishikesh via Mark Miller) + +* SOLR-10137: Ensure that ConfigSets created via API are mutable. (Hrishikesh via Mark Miller) + +* SOLR-10829: Fixed IndexSchema to enforce that uniqueKey can not be Points based for correctness (hossman) + +* SOLR-10836: The query parsers igain, significantTerms, and tlogit (used by streaming expressions by + the same name) might throw a NullPointerException if the referenced field had no indexed data in some + shards. The fix included an optimization to use Solr's cached AtomicReader instead of re-calculating. + (David Smiley) + +* SOLR-10715: /v2/ should not be an alias for /v2/collections (Cao Manh Dat) + +* SOLR-10835: Add support for point fields in Export Handler (Tomás Fernández Löbbe) + +* SOLR-10704: REPLACENODE may cause data loss when replicationFactor is 1. (ab, shalin) + +* SOLR-10833: Point numeric fields should throw SolrException(BAD_REQUEST) for malformed numbers in queries. + Trie numeric fields should throw SolrException(BAD_REQUEST) for malformed docValues range queries. + (hossman, Tomás Fernández Löbbe) + +* SOLR-10832: Fixed VersionInfo.getMaxVersionFromIndex when using PointsField with indexed="true" (hossman) + +* SOLR-10763: Admin UI replication tab sometimes empty when failed replications (janhoy, Bojan Vitnik) + +* SOLR-10824: fix NPE ExactSharedStatsCache, fixing maxdocs skew for terms which are absent at one of shards +when using one of Exact*StatsCache (Mikhail Khludnev) + +* SOLR-10963: Fix example json in MultipleAdditiveTreesModel javadocs. + (Stefan Langenmaier via Christine Poerschke) + +* SOLR-10914: RecoveryStrategy's sendPrepRecoveryCmd can get stuck for 5 minutes if leader is unloaded. (shalin) + +* SOLR-11198: downconfig downloads empty file as folder (Erick Erickson) + +Optimizations +---------------------- + +* SOLR-4509: Move to non deprecated HttpClient impl classes to remove stale connection + check on every request and move connection lifecycle management towards the client. + (Ryan Zezeski, Mark Miller, Shawn Heisey, Steve Davids) + +* SOLR-9255: Rename SOLR_AUTHENTICATION_CLIENT_CONFIGURER -> SOLR_AUTHENTICATION_CLIENT_BUILDER (janhoy) + +* SOLR-9579: Make Solr's SchemaField implement Lucene's IndexableFieldType, removing the + creation of a Lucene FieldType every time a field is indexed. (John Call, yonik) + +* SOLR-9822: JSON Facet API: Recover performance lost due to the DocValues transition to + an iterator API (LUCENE-7407). This only fixes calculating counts for single-valued + string fields from the FieldCache, resulting in up to 56% better throughput for those cases. + (yonik) + +* SOLR-10727: Avoid polluting the filter cache for certain types of faceting (typically ranges) when + the base docset is empty. (David Smiley) + +* SOLR-11070: Make docValues range queries behave the same as Trie/Point fields for Double/Float Infinity cases + (Tomás Fernández Löbbe, Andrey Kudryavtsev) + +* SOLR-10634: JSON Facet API: When a field/terms facet will retrieve all buckets (i.e. limit:-1) + and there are no nested facets, aggregations are computed in the first collection phase + so that the second phase which would normally involve calculating the domain for the bucket + can be skipped entirely, leading to large performance improvements. (yonik) + +* SOLR-10722: Speed up Solr's use of the UnifiedHighlighter be re-using FieldInfos. (David Smiley) + +Other Changes +---------------------- +* SOLR-10236: Removed FieldType.getNumericType(). Use getNumberType() instead. (Tomás Fernández Löbbe) + +* SOLR-10347: Removed index level boost support from "documents" section of the admin UI (Amrit Sarkar via + Tomás Fernández Löbbe) + +* SOLR-9959: SolrInfoMBean category and hierarchy cleanup. Per-component statistics are now obtained from + the metrics API, legacy JMX support has been replaced with SolrJmxReporter functionality. Several reporter + improvements (support for multiple prefix filters, "enabled" flag, reuse of service clients). (ab) + +* SOLR-10418: Expose safe system properties via metrics API as 'system.properties' in 'solr.jvm' group. + Add support for selecting specific properties from any compound metric using 'property' parameter to + /admin/metrics handler. (ab) + +* SOLR-10557: Make "compact" format default for /admin/metrics. (ab) + +* SOLR-10310: By default, stop splitting on whitespace prior to analysis + in edismax and standard/"lucene" query parsers. (Steve Rowe) + +* SOLR-10647: Move the V1 <-> V2 API mapping to SolrJ (noble) + +* SOLR-10584: We'll now always throw an exception if defaultOperator is found in schema. This config has + been deprecated since 3.6, and enforced for new configs since 6.6 (janhoy) + +* SOLR-10585: We'll now always throw an exception if defaultSearchField is found in schema. This config has + been deprecated since 3.6, and enforced for new configs since 6.6 (janhoy, David Smiley) + +* SOLR-10414: RecoveryStrategy is now a Runnable instead of a Thread. (Tomás Fernández Löbbe) + +* SOLR-10042: Delete old deprecated Admin UI, leaving the AngularJS UI the only one supported (janhoy) + +* SOLR-10378: Clicking Solr logo on AdminUI shows blank page (Takumi Yoshida via janhoy) + +* SOLR-10700: Deprecated and converted the PostingsSolrHighlighter to extend UnifiedSolrHighlighter and thus no + longer use the PostingsHighlighter. It should behave mostly the same. (David Smiley) + +* SOLR-10710: Fix LTR failing tests. (Diego Ceccarelli via Tomás Fernández Löbbe) + +* SOLR-10755: delete/refactor many solrj deprecations (hossman) + +* SOLR-10752: replicationFactor (nrtReplicas) default is 0 if tlogReplicas is specified when creating a collection + (Tomás Fernández Löbbe) + +* SOLR-10757: delete/refactor/cleanup CollectionAdminRequest deprecations (hossman) + +* SOLR-10793: BlobHandler should have a well-known permission name (noble) + +* SOLR-10744: Update noggit to newer version (0.8) (noble) + +* SOLR-10792: Deprecate and remove implicit registration of "lucenePlusSort" aka OldLuceneQParser (hossman) + +* SOLR-10791: Remove deprecated options in SSLTestConfig (hossman) + +* LUCENE-7852: Correct copyright year(s) in solr/LICENSE.txt file. + (Christine Poerschke, Steve Rowe) + +* SOLR-8668: In solrconfig.xml remove <mergePolicy> (and related <mergeFactor> and <maxMergeDocs>) + support in favor of the <mergePolicyFactory> element introduced by SOLR-8621 in Solr 5.5.0. + (Christine Poerschke, hossman) + +* SOLR-10799: Extracted functionality to collect eligible replicas from HttpShardHandler.prepDistributed() + to a new method (Domenico Fabio Marino via Tomás Fernández Löbbe) + +* SOLR-10801: Remove several deprecated methods that were exposed to plugin writers (hossman) + +* SOLR-10671: Add abstract doInit method to the SolrMetricReporter base class. + (Christine Poerschke, Anshum Gupta) + +* SOLR-10713: Ignore .pid and .out files in solr working directory (Jason Gerlowski via Mike Drob) + +* SOLR-10764: AutoScalingHandler should validate policy and preferences before updating zookeeper. (shalin) + +* SOLR-10782: Improve error handling and tests for Snitch and subclasses and general cleanups. (Noble Paul, shalin) + +* SOLR-10419: All collection APIs should use the new Policy framework for replica placement. (Noble Paul, shalin) + +* SOLR-10800: Factor out HttpShardHandler.transformReplicasToShardUrls from HttpShardHandler.prepDistributed. + (Domenico Fabio Marino, Christine Poerschke) + +* SOLR-10779: JavaBinCodec should use close consistently rather than having marshal() and close() call finish() + (which closes the underlying stream). (Erick Erickson) + +* SOLR-9623: Disable remote streaming in example configs by default. Adjust Upload Limit defaults (janhoy, yonik) + +* SOLR-4646: eDismax lowercaseOperators now defaults to "false" for luceneMatchVersion >= 7.0.0 (janhoy, David Smiley) + +* SOLR-8256: Set legacyCloud=false as default (Cao Manh Dat) + +* SOLR-10929: Removed unused 'valType' option from ExternalFileField (hossman) + +* SOLR-10864: Simplified how Trie vs Points based numerics are randomized by SolrTestCaseJ4 by adding a static + option to PointsField to ignore 'precisionStep' attribute. This change also begins to attempt to randomize + 'docValues' on numeric field types unless tests explicity enable/disable them. (hossman, Steve Rowe) + +* LUCENE-7883: Solr no longer uses the context class loader when resolving + resources, they are only resolved against Solr's own or "core" class loader + by default. (Uwe Schindler) + +* SOLR-10915: Make builder based SolrClient constructors to be the only valid way to construct client objects and + increase the visibility of builder elements to be protected so extending the builder, and the clients is possible. + (Jason Gerlowski, Anshum Gupta) + +* SOLR-10823: Add reporting period to SolrMetricReporter base class. (Christine Poerschke) + +* SOLR-10807: Randomize Points based numeric field types in all existing tests & test schemas + where Trie numerics were previously hardcoded. (hossman, Steve Rowe, Anshum Gupta) + +* SOLR-6807: Changed requestDispatcher's handleSelect to default to false, thus ignoring "qt". + Simplified configs to not refer to handleSelect or "qt". Switch all tests that assumed true to assume false + (added leading '/' in request handlers). Switch all tests referring to "standard" request handler to + instead refer to "/select" with SearchHandler. Deprecated the old StandardRequestHandler. (David Smiley) + +* SOLR-10456: Deprecate timeout related setters from SolrClients, and replace with Builder based implementation + (Jason Gerlowski, Anshum Gupta) + +* SOLR-11004: Consolidate SolrClient builder code in an abstract base class (Jason Gerlowski, Anshum Gupta) + +* SOLR-10967: Cleanup the default configset. The changes involve some documentation corrections, removing the currency + field from the examples and a few dynamic fields (Varun Thacker) + +* SOLR-11016: Fix TestCloudJSONFacetJoinDomain test-only bug (hossman) + +* SOLR-11021: The elevate.xml config-file is made optional in the ElevationComponent. + The default configset doesn't ship with a elevate.xml file anymore (Varun Thacker) + +* SOLR-10898: Fix SOLR-10898 to not deterministicly fail 1/512 runs (hossman) + +* SOLR-10796: TestPointFields: increase randomized testing of non-trivial values. (Steve Rowe) + +* SOLR-11068: MOVEREPLICA and REPLACENODE API parameter names are now 'sourceNode' and 'targetNode'. The old names + viz. 'fromNode' for MOVEREPLICA and 'source', 'target' for REPLACENODE have been deprecated. (shalin) + +* SOLR-11088: Fix sporadic failures of MetricsHandlerTest.testPropertyFilter on jenkins (shalin) + +* SOLR-11037: Refactor to provide NodeConfig.getSolrDataHome internal API. (ab, janhoy, shalin) + +* SOLR-11119: Switch from Trie to Points field types in the .system collection schema. (Steve Rowe) + +* SOLR-10494: Make default response format JSON (wt=json), and also indent text responses formats + (indent=on) by default (Trey Grainger & Cassandra Targett via hossman) + +* SOLR-10760,SOLR-11315,SOLR-11313: Remove trie field types and fields from example schemas. (Steve Rowe) + +* SOLR-11056: Add random range query test that compares results across Trie*, *Point and DocValue-only fields + (Tomás Fernández Löbbe) + +* SOLR-10926: Increase the odds of randomly choosing point fields in our SolrTestCaseJ4 numeric type randomization. + (hossman, Steve Rowe) + +* SOLR-10846: ExternalFileField/FloatFieldSource should throw a clear exception on initialization with + a Points-based keyField, which is not supported. (hossman, Steve Rowe) + +* SOLR-11155: /analysis/field and /analysis/document requests should support points fields. + (Jason Gerlowski, Steve Rowe) + +* SOLR-10756: Undeprecate ZkStateReader.updateClusterState(), mark as @lucene.internal, and rename to + forciblyRefreshAllClusterStateSlow(). (hossman, shalin, Steve Rowe) + +* SOLR-11036: Separately report disk space metrics for solr.data.home and core root directory. (ab) + +* SOLR-10919: ord & rord functions give confusing errors with PointFields. (hossman, Steve Rowe) + +* SOLR-10847: Provide a clear exception when attempting to use the terms component with points fields. + (hossman, Steve Rowe) + +* SOLR-9321: Remove deprecated methods of ClusterState. (Jason Gerlowski, ishan, Cao Manh Dat) + +* SOLR-10033: When attempting to facet with facet.mincount=0 over points fields, raise mincount to 1 + and log a warning. (Steve Rowe) + +* SOLR-11178: Change error handling in AutoScalingHandler to be consistent w/ other APIs (noble) + +* SOLR-11023: Added EnumFieldType, a non-Trie-based version of EnumField, and deprecated EnumField + in favor of EnumFieldType. (hossman, Steve Rowe) + +* SOLR-10803: Mark all Trie/LegacyNumeric based fields @deprecated in Solr7. (Steve Rowe) + +* SOLR-10821: Ref guide documentation for Autoscaling (Noble Paul, Cassandra Targett, shalin) + +* SOLR-11239: A special value of -1 can be specified for 'maxShardsPerNode' to denote that + there is no limit. The bin/solr script send maxShardsPerNode=-1 when creating collections. The + use of maxShardsPerNode is not supported when a cluster policy is in effect or when a + collection specific policy is specified during collection creation. + (Noble Paul, shalin) + +* SOLR-11183: V2 APIs are now available at /api endpoint. (Ishan Chattopadhyaya) + +* SOLR-10617: JDBCStream accepts columns of type TIME, DATE & TIMESTAMP as well as CLOBs and decimal + numeric types (James Dyer) + +* SOLR-10400: Replace (instanceof TrieFooField || instanceof FooPointField) constructs with + FieldType.getNumberType() or SchemaField.getSortField() where appropriate. (hossman, Steve Rowe) + +* SOLR-10438: Assign explicit useDocValuesAsStored values to all points field types in + schema-point.xml/TestPointFields. (hossman, Steve Rowe) + +* LUCENE-7705: Allow CharTokenizer-derived tokenizers and KeywordTokenizer to configure the max token length. + (Amrit Sarkar via Erick Erickson) + +* SOLR-10659: Remove ResponseBuilder.getSortSpec use in SearchGroupShardResponseProcessor. + (Judith Silverman via Christine Poerschke) + +* SOLR-10741: Factor out createSliceShardsStr method from HttpShardHandler.prepDistributed. + (Domenico Fabio Marino via Christine Poerschke) + +* SOLR-10790: Fix warnings in Assign.java and ReplicaAssigner.java classes. + (Christine Poerschke) + +* SOLR-8437: Improve RAMDirectory details in sample solrconfig files (Mark Miller, Varun Thacker) + +* SOLR-8762: return child docs in DIH debug (Gopikannan Venugopalsamy via Mikhail Khludnev) + +* SOLR-10501: Test sortMissing{First,Last} with points fields. (Steve Rowe) + +* SOLR-10761: Switch trie numeric/date fields to points in data-driven-enabled example and test schemas. + (Steve Rowe) + +* SOLR-10851: SolrClients should clarify expectations for solrServerUrl parameter (Jason Gerlowski + via Tomás Fernández Löbbe) + +* SOLR-10891: BBoxField should support point-based number sub-fields. (Steve Rowe) + +* SOLR-10834: Fixed tests and test configs to stop using numeric uniqueKey fields (hossman) + +* SOLR-10883: Ref guide: Escape replacement substitutions, e.g. => to right arrow, so that they are + rendered visibly in the PDF. Also add .adoc file checks to the top-level validate target, including + for the invisible substitutions PDF problem. (Steve Rowe) + +* SOLR-10503,SOLR-10502: Deprecate CurrencyField in favor of new CurrencyFieldType, which works + with point fields and provides control over dynamic fields used for the raw amount and currency + code sub-fields. (hossman, Steve Rowe) + +* SOLR-11261, SOLR-10966: Upgrade to Hadoop 2.7.4 to fix incompatibility with Java 9. + (Uwe Schindler) + +* SOLR-11324: Clean up mention of trie fields in documentation and source comments. (Steve Rowe) + +================== 6.6.5 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.10 +Jetty 9.3.14.v20161028 + +New Features +---------------------- + +* SOLR-12530: Ability to disable configset upload via -Dconfigset.upload.enabled=false startup parameter + (Ishan Chattopadhyaya) + +Bug Fixes +---------------------- + +* SOLR-12450: Don't allow referal to external resources in various config files (CVE-2018-8026). + (Yuyang Xiao, Uwe Schindler) + +================== 6.6.4 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.17 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.11 +Jetty 9.4.10.v20180503 + +Bug Fixes +---------------------- + +* SOLR-12316: Do not allow to use absolute URIs for including other files in solrconfig.xml and schema parsing. + (Ananthesh, Ishan Chattopadhyaya, Uwe Schindler) + +================== 6.6.3 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.17 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.11 +Jetty 9.4.8.v20171121 + +Bug Fixes +---------------------- + +* SOLR-11971: Don't allow referal to external resources in DataImportHandler's dataConfig request parameter. + (麦 香浓郁, Uwe Schindler) + +* SOLR-11503: Collections created with legacyCloud=true cannot be opened if legacyCloud=false (Erick Erickson) + +* SOLR-11993: LeaderInitiatedRecoveryThread does not retry on UnknownHostException (shalin, Cao Manh Dat, Steve Rowe) + +================== 6.6.2 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.16 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.10 +Jetty 9.3.14.v20161028 + +Upgrade Notes +---------------------- + +* SOLR-11477: in the XML query parser (defType=xmlparser or {!xmlparser ... }) + the resolving of external entities is now disallowed by default. + +* SOLR-11482: RunExecutableListener was deprecated and is disabled by default for + security reasons. Legacy applications still using it must explicitely pass + '-Dsolr.enableRunExecutableListener=true' to the Solr command line. + Be aware that you should really disable API-based config editing at the same + time, using '-Ddisable.configEdit=true'! (Uwe Schindler) + +Bug Fixes +---------------------- + +* SOLR-11477: Disallow resolving of external entities in the XML query parser (defType=xmlparser). + (Michael Stepankin, Olga Barinova, Uwe Schindler, Christine Poerschke) + + +* SOLR-11297: Message "Lock held by this virtual machine" during startup. Solr is trying to start some cores twice. + (Luiz Armesto, Shawn Heisey, Erick Erickson) + +Other Changes +---------------------- + +* SOLR-10335: Upgrade to Tika 1.16. (Tim Allison, Tomás Fernández Löbbe via shalin) + +================== 6.6.1 ================== + +Bug Fixes +---------------------- + +* SOLR-10857: standalone Solr loads UNLOADed core on request (Erick Erickson, Mikhail Khludnev) + +* SOLR-11024: ParallelStream should set the StreamContext when constructing SolrStreams (Joel Bernstein) + +* SOLR-10908: CloudSolrStream.toExpression incorrectly handles fq clauses (Rohit via Erick Erickson) + +* SOLR-11177: CoreContainer.load needs to send lazily loaded core descriptors to the proper list rather than send + them all to the transient lists. (Erick Erickson) + +* SOLR-11122: Creating a core should write a core.properties file first and clean up on failure + (Erick Erickson) + +* SOLR-10910: Clean up a few details left over from pluggable transient core and untangling + CoreDescriptor/CoreContainer references (Erick Erickson) + +* SOLR-10721: Provide a way to know when Core Discovery is finished and when all async cores are done loading + (Erick Erickson) + +* SOLR-11069: CDCR bootstrapping can get into an infinite loop when a core is reloaded (Amrit Sarkar, Erick Erickson) + +* SOLR-11221: SolrJmxReporter broken on core reload. This resulted in some or most metrics not being reported + via JMX after core reloads, depending on timing. (ab) + +* SOLR-11261, SOLR-10966: Upgrade to Hadoop 2.7.4 to fix incompatibility with Java 9. + (Uwe Schindler) + +* SOLR-11228: Exclude static html files in the partials directory from authentication and authorization checks. The UI + will open correctly with kerberos enabled (Ishan Chattopadhyaya, Varun Thacker) + +================== 6.6.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.10 +Jetty 9.3.14.v20161028 + +Upgrade Notes +---------------------- + +* Solr contribs map-reduce, morphlines-core and morphlines-cell have been removed. + +* JSON Facet API now uses hyper-log-log for numBuckets cardinality calculation and + calculates cardinality before filtering buckets by any mincount greater than 1. + +* ZooKeeper dependency has been upgraded from 3.4.6 to 3.4.10. + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-9992: Add support for grouping with PointFIelds. (Cao Manh Dat) + +* SOLR-9994: Add support for CollapseQParser with PointFields. (Varun Thacker, Cao Manh Dat) + +* SOLR-10076: Hide keystore and truststore passwords from /admin/info/* outputs. (Mano Kovacs via Mark Miller) + +* SOLR-6736: Adding support for uploading zipped configsets using ConfigSets API (Varun Rajput, Ishan Chattopadhyaya, + Noble Paul, Anshum Gupta, Gregory Chanan) + +* SOLR-10349: Add totalTermFreq support to TermsComponent. (Shai Erera) + +* SOLR-9993: Add support for ExpandComponent with PointFields. (Cao Manh Dat) + +* SOLR-10239: MOVEREPLICA API (Cao Manh Dat, Noble Paul, shalin) + +* SOLR-9936: Allow configuration for recoveryExecutor thread pool size. (Tim Owen via Mark Miller) + +* SOLR-10447: Collections API now supports a LISTALIASES command to return a list of all collection aliases. + (Yago Riveiro, Ishan Chattopadhyaya, Mark Miller, Steve Molloy, Shawn Heisey, Mike Drob, janhoy) + +* SOLR-10446: CloudSolrClient can now be initialized using the base URL of a Solr instance instead of + ZooKeeper hosts. This is possible through the use of newly introduced HttpClusterStateProvider. + To fetch a list of collection aliases, this depends on LISTALIASES command, and hence this way of + initializing CloudSolrClient would not work if you have collection aliases on older versions of Solr + server that doesn't support LISTALIASES. (Ishan Chattopadhyaya, Noble Paul) + +* SOLR-10082: Variance and Standard Deviation aggregators for the JSON Facet API. + Example: json.facet={x:"stddev(field1)", y:"variance(field2)"} + (Rustam Hashimov, yonik) + +* SOLR-10505: Add multi-field support to TermsComponent when requesting terms' statistics. (Shai Erera) + +* SOLR-10537: SolrJ: Added SolrParams.toLocalParamsString() and ClientUtils.encodeLocalParamVal. (David Smiley) + +* SOLR-10507: Core Admin status command to emit collection details of each core (noble) + +* SOLR-10521: introducing sort=childfield(field) asc for searching by {!parent} (Mikhail Khludnev) + +* SOLR-9596: Add Solr support for SimpleTextCodec, via <codecFactory class="solr.SimpleTextCodecFactory"/> + in solrconfig.xml (per-field specification in the schema is not possible). (Steve Rowe) + +* SOLR-1485: Add payload support with payload() value source and {!payload_score} and {!payload_check} + query parsers. (Erik Hatcher) + +* SOLR-10430: Add ls command to ZkCLI for listing only sub-directories. (Peter Szantai-Kis via Mark Miller) + +* SOLR-10583: JSON Faceting now supports a query time 'join' domain change option (hossman) + +* SOLR-9530: An Update Processor to convert normal update operation to an atomic operations such as + add, set,inc, remove ,set, removeregex (Amrit Sarkar, noble) + +* SOLR-10303: Add date/time Stream Evaluators (Gethin James, Dennis Gove, Joel Bernstein) + +* SOLR-10351: Add analyze Stream Evaluator to support streaming NLP (Joel Bernstein) + +* SOLR-10426: Add shuffle Streaming Expression (Joel Bernstein) + +* SOLR-10274: The search Streaming Expression should work in non-SolrCloud mode (Joel Bernstein) + +* SOLR-10504: Add echo Streaming Expression (Joel Bernstein) + +* SOLR-10516: Add eval Streaming Expression (Joel Bernstein) + +* SOLR-10566: Add timeseries Streaming Expression (Joel Bernstein) + +* SOLR-10559: Add let, get and tuple Streaming Expressions (Dennis Gove, Joel Bernstein) + +* SOLR-10582: Add Correlation Stream Evaluator (Joel Bernstein) + +* SOLR-10536: stats Streaming Expression should work in non-SolrCloud mode (Joel Bernstein) + +* SOLR-10622: Add regress and predict Stream Evaluators (Joel Bernstein) + +* SOLR-10626: Add covariance Stream Evaluator (Joel Bernstein) + +* SOLR-10625: Add convolution Stream Evaluator (Joel Bernstein) + +* SOLR-10638: Add normalize Stream Evaluator (Joel Bernstein) + +* SOLR-8440: Support for enabling basic authentication using bin/solr|bin/solr.cmd. (Ishan Chattopadhyaya, janhoy, + Noble Paul, Hrishikesh Gadre) + +* SOLR-10292: Adds CartesianProductStream which turns a single tuple with a multi-valued field into N + tuples, one for each value in the multi-valued field. (Dennis Gove) + +Optimizations +---------------------- + +* SOLR-9184: Add a static convenience method ModifiableSolrParams#of(SolrParams) which returns the same + instance if it already is modifiable, otherwise creates a new ModifiableSolrParams instance. + (Jörg Rathlev via Koji) + +* SOLR-10499: facet.heatmap is now significantly faster when the docset (base query) matches everything and there are no + deleted docs. It's also faster when the docset matches a small fraction of the index or none. (David Smiley) + +* SOLR-9217: Reduced heap consumption for filter({!join ... score=...}) + (Andrey Kudryavtsev, Gopikannan Venugopalsamy via Mikhail Khludnev) + +* SOLR-10548: JSON Facet API now uses hyper-log-log++ for determining the number of buckets + when merging requests from a multi-shard distributed request. (yonik) + +* SOLR-10524: Better ZkStateWriter batching (Cao Manh Dat, Noble Paul, shalin, Scott Blum) + +* SOLR-10619: Optimize using cache for DistributedQueue in case of single-consumer (Cao Manh Dat, Scott Blum) + +Bug Fixes +---------------------- +* SOLR-10281: ADMIN_PATHS is duplicated in two places and inconsistent. This can cause automatic + retries to /admin/metrics handler by the CloudSolrClient. (shalin) + +* SOLR-10108: bin/solr script recursive copy broken (Erick Erickson) + +* SOLR-10362: "Memory Pool not found" error when reporting JVM metrics. (ab) + +* SOLR-10369: bin\solr.cmd delete and healthcheck now works again; fixed continuation chars ^ (Luis Goes via janhoy) + +* SOLR-10387: zkTransfer normalizes destination path incorrectly if source is a windows directory + (gopikannan venugopalsamy, Erick Erickson) + +* SOLR-10323: fix to SpellingQueryConverter to properly strip out colons in field-specific queries. + (Amrit Sarkar via James Dyer) + +* SOLR-10264: Fixes multi-term synonym parsing in ManagedSynonymFilterFactory. + (Jörg Rathlev, Steve Rowe, Christine Poerschke) + +* SOLR-8807: fix Spellcheck "collateMaxCollectDocs" parameter to work with queries that have the + CollpasingQParserPlugin applied. (James Dyer) + +* SOLR-10474: TestPointFields.testPointFieldReturn() depends on order of unsorted hits. (Steve Rowe) + +* SOLR-10473: Correct LBHttpSolrClient's confusing SolrServerException message when timeAllowed is exceeded. + (Christine Poerschke) + +* SOLR-10047: Mismatched Docvalues segments cause exception in Sorting/Faceting. Solr now uninverts per segment + to avoid such exceptions. (Keith Laban via shalin) + +* SOLR-10472: Fixed uninversion (aka: FieldCache) bugs with the numeric PointField classes, and CurrencyField (hossman) + +* SOLR-5127: Multiple highlight fields and wildcards are now supported e.g. hl.fl=title,text_* + (Sven-S. Porst, Daniel Debray, Simon Endele, Christine Poerschke) + +* SOLR-10493: Investigate SolrCloudExampleTest failures. (Erick Erickson) + +* SOLR-10552: JSON Facet API numBuckets was not consistent between distributed and non-distributed requests + when there was a mincount > 1. This has been corrected by changing numBuckets cardinality processing to + ignore mincount > 1 for non-distributed requests. (yonik) + +* SOLR-10520: child.facet.field doubled counts at least when rows>0. (Dr. Oleg Savrasov via Mikhail Khludnev) + +* SOLR-10480: Full pagination in JSON Facet API using offset does not work. (yonik) + +* SOLR-10526: facet.heatmap didn't honor facet exclusions ('ex') for distributed search. (David Smiley) + +* SOLR-10500: nested child docs are adopted by neighbour when several parents come in update/json/docs + (Alexey Suprun,noble via Mikhail Khludnev) + +* SOLR-10316: Unloading a core can remove a ZK SolrCore registration entry for the wrong SolrCore. (Mark Miller) + +* SOLR-10588: Prevent redundant core reload on config update (Mikhail Khludnev) + +* SOLR-10549: The new 'large' attribute had been forgotten in /schema/fieldtypes?showDefaults=true (David Smiley) + +* SOLR-10615: requests are suspended until SolrDispatchFilter initialization is completed. + After core container shutdown or severe initialization problem Solr responds with + http stauts 404 Not Found instead of 500 as it was before (Mikhail Khludnev) + +* SOLR-8149: Admin UI - Plugins / Stats - active item is now highlighted (Labuzov Dmitriy via janhoy) + +* SOLR-10630: HttpSolrCall.getAuthCtx().new AuthorizationContext() {...}.getParams() + sometimes throws java.lang.NullPointerException (hu xiaodong via shalin) + +* SOLR-9527: Improve distribution of replicas when restoring a collection + (Hrishikesh Gadre, Stephen Lewis, Rohit, Varun Thacker) + +* LUCENE-7821: The classic and flexible query parsers, as well as Solr's + "lucene"/standard query parser, should require " TO " in range queries, + and accept "TO" as endpoints in range queries. (hossman, Steve Rowe) + +* SOLR-10735: Windows script (solr.cmd) didn't work properly with directory containing spaces. Adding quotations + to fix (Uwe Schindler, janhoy, Tomas Fernandez-Lobbe, Ishan Chattopadhyaya) + +Ref Guide +---------------------- + +* SOLR-10758: Modernize the Solr ref guide's Chinese language analysis coverage. (Steve Rowe) + +Other Changes +---------------------- + +* SOLR-9221: Remove Solr contribs: map-reduce, morphlines-core and morphlines-cell. (Steve Rowe) + +* SOLR-10249: Refactor IndexFetcher.doFetch() to return a more detailed result. (Jeff Miller via David Smiley) + +* SOLR-10304: Refactor Document handling out of SolrIndexSearcher into a new class "SolrDocumentFetcher". + Deprecated SolrPluginUtils.docListToSolrDocumentList(). (David Smiley) + +* SOLR-10352: bin/solr script now prints warning when available system entropy is lower than 300 (Esther Quansah via + Ishan Chattopadhyaya) + +* SOLR-10344: Update Solr default/example and test configs to use WordDelimiterGraphFilterFactory. (Steve Rowe) + +* SOLR-10343: Update Solr default/example and test configs to use SynonymGraphFilterFactory. (Steve Rowe) + +* SOLR-10365: Handle a SolrCoreInitializationException while publishing core state during SolrCore creation + (Ishan Chattopadhyaya) + +* SOLR-10357: Enable edismax and standard query parsers to handle the option combination + sow=false / autoGeneratePhraseQueries="true" by setting QueryBuilder.autoGenerateMultiTermSynonymsQuery. + (Steve Rowe) + +* SOLR-10147: Admin UI -> Cloud -> Graph: Impossible to see shard state (Amrit Sarkar, janhoy) + +* SOLR-10399: Generalize some internal facet logic to simplify points/non-points field handling (Adrien Grand, hossman) + +* SOLR-7383: New DataImportHandler 'atom' example, replacing broken 'rss' example (Alexandre Rafalovitch) + +* SOLR-9601: Redone DataImportHandler 'tika' example, removing all unused and irrelevant definitions (Alexandre Rafalovitch) + +* SOLR-8906: Make transient core cache pluggable (Erick Erickson) + +* SOLR-9745: print errors from solr.cmd (Gopikannan Venugopalsamy via Mikhail Khludnev) + +* SOLR-10394: Rename getSortWithinGroup to getWithinGroupSort in search.grouping.Command class. + (Judith Silverman, Christine Poerschke) + +* SOLR-10440: LBHttpSolrClient.doRequest is now always wrapped in a Mapped Diagnostic Context (MDC). + (Christine Poerschke) + +* SOLR-10429: UpdateRequest#getRoutes()should copy the response parser (noble) + +* SOLR-10007: Clean up references to CoreContainer and CoreDescriptors (Erick Erickson) + +* SOLR-10151: Use monotonically incrementing counter for doc ids in TestRecovery. (Peter Szantai-Kis, Mano Kovacs via Mark Miller) + +* SOLR-10514: Upgrade Metrics library to 3.2.2. (ab) + +* SOLR-9386: Upgrade Zookeeper to 3.4.10. (Shawn Heisey, Steve Rowe) + +* SOLR-10519: SolrCLI.atPath cannot handle children that begin with a slash. (Erick Erickson) + +* SOLR-9867: Adding isLoading=true as core status. Fixing start after stop scenario in bin/solr + (Andrey Kudryavtsev, Mikhail Khludnev) + +* SOLR-7041: Cutover tests to using 'q.op' and 'df' instead of schema 'defaultOperator' and 'defaultSearchField' (janhoy) + +* SOLR-10601: StreamExpressionParser should handle white space around = in named parameters (Joel Bernstein) + +* SOLR-10614: Static fields have turned to instance's field in SimplePostTool. + Enabled TestSolrCLIRunExample.testTechproductsExample(). (Andrey Kudryavtsev, Mikhail Khludnev) + +* SOLR-10522: Revert SpellCheckComponent response format change from SOLR-9972 (rel. 6.5.0). While this + was an improvement for the json "arrntv" format, it caused problems for the default json format. + (James Dyer, reported by Nikita Pchelintsev) + +* SOLR-10644: solr.in.sh installed by install script should be writable by solr user (janhoy) + +* SOLR-10729: Deprecated LatLonType, GeoHashField, SpatialPointVectorFieldType, and SpatialTermQueryPrefixTreeFieldType. + Instead, switch to LatLonPointSpatialField or SpatialRecursivePrefixTreeFieldType or RptWithGeometrySpatialField. + (David Smiley) + +================== 6.5.1 ================== + +Bug Fixes +---------------------- + +* SOLR-10383: Fix debug related NullPointerException in solr/contrib/ltr OriginalScoreFeature class. + (Vitezslav Zak, Christine Poerschke) + +* SOLR-10416: The JSON output of /admin/metrics is fixed to write the container as a + map (SimpleOrderedMap) instead of an array (NamedList). (shalin) + +* SOLR-10277: On 'downnode', lots of wasteful mutations are done to ZK. + (Joshua Humphries, Scott Blum, Varun Thacker, shalin) + +* SOLR-10421: Fix params persistence for solr/contrib/ltr (MinMax|Standard)Normalizer classes. + (Jianxiong Dong, Christine Poerschke) + +* SOLR-10404: The fetch() streaming expression wouldn't work if a value included query syntax chars (like :+-). + Fixed, and enhanced the generated query to not pollute the queryCache. (David Smiley) + +* SOLR-10423: Disable graph query production via schema configuration <fieldtype ... enableGraphQueries="false">. + This fixes broken queries for ShingleFilter-containing query-time analyzers when request param sow=false. + (Steve Rowe) + +* SOLR-10425: Fix indexed="false" on numeric PointFields (Tomás Fernández Löbbe, hossman) + +* SOLR-10341: SQL AVG function mis-interprets field type. (Joel Bernstein) + +* SOLR-10444: SQL interface does not use client cache. (Joel Bernstein) + +* SOLR-10420: Solr 6.x leaking one SolrZkClient instance per second (Scott Blum, Cao Manh Dat, Markus Jelsma, Steve Rowe) + +* SOLR-10439: The new 'large' attribute had been forgotten in /schema/fields?showDefaults=true (David Smiley) + +* SOLR-10527: edismax with sow=false fails to create dismax-per-term queries when any field is boosted. + (Steve Rowe) + +================== 6.5.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.14.v20161028 +Apache Calcite 1.11.0 + +Detailed Change List +---------------------- + +Upgrade Notes +---------------------- +* SOLR-10226: JMX metric "avgTimePerRequest" (and the corresponding metric in the metrics API for + each handler) used to be a simple non-decaying average based on total cumulative time and the + number of requests. New Codahale Metrics implementation applies exponential decay to this value, + which heavily biases the average towards the last 5 minutes. (ab) + +* SOLR-8593: Parallel SQL now uses Apache Calcite as its SQL framework. As part of this change + the default aggregation mode has been changed to facet rather than map_reduce. There has also been changes + to the SQL aggregate response and some SQL syntax changes. Consult the documentation for full details. + + +New Features +---------------------- + +* SOLR-9836: Add ability to recover from leader when index corruption is detected on SolrCore creation. + (Mike Drob via Mark Miller) + +* SOLR-9926: Allow passing arbitrary java system properties to zkcli. (Hrishikesh Gadre via Mark Miller) + +* SOLR-9885: Allow pre-startup Solr log management in Solr bin scripts to be disabled. (Mano Kovacs via Mark Miller) + +* SOLR-9481: Authentication and Authorization plugins now work in standalone mode if security.json is placed in + SOLR_HOME on every node. Editing config through API is supported but affects only that one node. (janhoy) + +* SOLR-8029: Added new style APIs and a framework for creating new APIs and mapping old APIs to new + (noble, Steve Rowe, Cassandra Targett, Timothy Potter) + +* SOLR-9933: SolrCoreParser now supports configuration of custom SpanQueryBuilder classes. + (Daniel Collins, Christine Poerschke) + +* SOLR-7955: Auto create .system collection on first request if it does not exist (noble) + +* SOLR-10087: StreamHandler now supports registering custom streaming expressions from the blob store (Kevin Risden) + +* SOLR-9997: Enable configuring SolrHttpClientBuilder via java system property. (Hrishikesh Gadre via Mark Miller) + +* SOLR-9912: Add facet.excludeTerms parameter support. (Jonny Marks, David Smiley, Christine Poerschke) + +* SOLR-9916: Adds Stream Evaluators to support evaluating values from tuples. Supports boolean, + numeric, and conditional evaluators. BooleanOperations have been removed in preference of + BooleanEvaluators. (Dennis Gove) + +* SOLR-9903: Stop interrupting the update executor on shutdown, it can cause graceful shutdowns to put replicas into Leader + Initiated Recovery among other undesirable things. (Mark Miller) + +* SOLR-8396: Add support for PointFields in Solr (Ishan Chattopadhyaya, Tomás Fernández Löbbe) + +* SOLR-9987: Add support for MultiValued DocValues in PointFields using SortedNumericDocValues (Tomás Fernández Löbbe) + +* SOLR-5944: In-place updates of Numeric DocValues. To leverage this, the _version_ field and the updated + field must both be stored=false, indexed=false, docValues=true. (Ishan Chattopadhyaya, hossman, noble, + shalin, yonik) + +* SOLR-10158: Add support for "preload" option in MMapDirectoryFactory. + (Amrit Sarkar via Uwe Schindler) + +* SOLR-10153 & SOLR-10152: The Unified and Postings based highlighters: Add hl.bs.type=SEPARATOR along with new param + hl.bs.separator to break passages by a provided single character. (Amrit Sarkar, David Smiley) + +* SOLR-10156: Add significantTerms Streaming Expression (Joel Bernstein) + +* SOLR-8593: Integrate Apache Calcite into the SQLHandler (Kevin Risden, Cao Manh Dat, Joel Bernstein) + +* SOLR-10146: Added button to the Admin UI 'Collection' tab for deleting an inactive shard (Amrit Sarkar, janhoy) + +* SOLR-9999: Instrument DirectUpdateHandler2. This registers existing statistics under metrics API and adds + more metrics to track the rates of update and delete commands. (ab) + +* SOLR-9986: Implement DatePointField (Cao Manh Dat, Tomás Fernández Löbbe) + +* SOLR-8045: Deploy V2 API at /v2 instead of /solr/v2 (Cao Manh Dat, Noble Paul) + +* SOLR-10039: New LatLonPointSpatialField replacement for LatLonType (and some uses of RPT). Multi-value capable + indexed geo lat-lon points, query by rect or circle. Efficient distance sorting/boosting too. (David Smiley) + +* SOLR-10250: CloudSolrClient can now return versions for documents added or deleted when "versions=true" is passed. + However, if there is a leader election while this request is in transit, the versions may not be returned from that + shard. (Boris Naguet, Ishan Chattopadhyaya) + +* SOLR-9045: Make RecoveryStrategy settings configurable. (Christine Poerschke) + +* SOLR-10224: Add disk total and disk free metrics. (ab) + +* SOLR-10085: SQL result set fields should be ordered by the field list (Joel Bernstein) + +* SOLR-10254: significantTerms Streaming Expression should work in non-SolrCloud mode (Joel Bernstein) + +* SOLR-10286: string/text fields may now declare themselves as large="true" in the schema. Large fields are always + lazy loaded and will only take up space in the document cache if the actual value is < 512KB. This option + requires "stored" and must not be multiValued. It's intended for fields that might have very large values so that + they don't get cached in memory. (David Smiley) + +* SOLR-9185: Solr's edismax and "Lucene"/standard query parsers will no longer split on whitespace before sending + terms to analysis, if given the "sow=false" request param ("sow"=>"split on whitespace"). This enables multi-term + source synonyms to match at query-time using SynonymGraphFilterFactory; other analysis components will also now + work at query time, e.g. ShingleFilterFactory. By default, and when the "sow=true" param is specified, these + parsers' behavior remains the same: queries will be split on whitespace before sending individual terms to analysis. + (Steve Rowe) + +Bug Fixes +---------------------- + +* SOLR-9976: Fix init bug in SegmentsInfoRequestHandlerTest (hossman) + +* SOLR-9977: Fix config bug in DistribDocExpirationUpdateProcessorTest that allowed false assumptions + about when index version changes (hossman) + +* SOLR-9979: Macro expansion should not be done in shard requests (Tomás Fernández Löbbe) + +* SOLR-9114: NPE using TermVectorComponent, MoreLikeThisComponent in combination with ExactStatsCache (Cao Manh Dat, Varun Thacker) + +* SOLR-10049: Collection deletion leaves behind the snapshot metadata (Hrishikesh Gadre via yonik) + +* SOLR-10120: A SolrCore reload can remove the index from the previous SolrCore during replication index rollover. (Mark Miller) + +* SOLR-10124: Replication can skip removing a temporary index directory in some cases when it should not. (Mark Miller) + +* SOLR-9996: Unstored IntPointField returns Long type (Ishan Chattopadhyaya) + +* SOLR-10104: BlockDirectoryCache release hooks do not work with multiple directories. (Mike Drob, Mark Miller) + +* SOLR-10121: Fix race conditions in HDFS BlockCache that can contribute to corruption in high + concurrency situations. (yonik) + +* SOLR-10141: Upgrade to Caffeine 2.4.0 since v1.0.1 contributed to BlockCache corruption because the + removal listener was called more than once for some items and not at all for other items. (Ben Manes, yonik) + +* SOLR-10114: Reordered delete-by-query causes inconsistenties between shards that have + child documents (Mano Kovacs, Mihaly Toth, yonik) + +* SOLR-10159: When DBQ is reordered with an in-place update, upon whose updated value the DBQ is based + on, the DBQ fails due to excessive caching in DeleteByQueryWrapper (Ishan Chattopadhyaya) + +* SOLR-10020: CoreAdminHandler silently swallows some errors. (Mike Drob via Erick Erickson) + +* SOLR-10063: CoreContainer shutdown has race condition that can cause a hang on shutdown. (Mark Miller) + +* SOLR-10170: ClassCastException in RecoveryStrategy. (Mark Miller) + +* SOLR-9846: Overseer is not always closed after being started. (Mark Miller) + +* SOLR-10168: ShardSplit can fail with NPE in OverseerCollectionMessageHandler#waitForCoreAdminAsyncCallToComplete. (Mark Miller) + +* SOLR-9824: Some bulk update paths could be very slow due to CUSC polling. (David Smiley, Mark Miller) + +* SOLR-10055: Linux installer now renames existing bin/solr.in.* as bin/solr.in.*.orig to make the installed config in + /etc/defaults be the one found by default when launching solr manually. (janhoy) + +* SOLR-10196: ElectionContext#runLeaderProcess can hit NPE on core close. (Mark Miller) + +* SOLR-10225: Fix HDFS BlockCache evictions metric to not count explicit removal + due to a directory close. (yonik) + +* SOLR-10088: Installer script does not put zoo.cfg in SOLR_HOME (janhoy) + +* SOLR-10226: add back "totalTime" metric to all handlers. See also the back-compat note. (ab) + +* SOLR-9838: "inc" atomic update doesn't respect default field value (hoss, Amrit Sarkar, Ishan Chattopadhyaya) + +* SOLR-10269: MetricsHandler JSON output incorrect. (ab) + +* SOLR-10279: The autoAddReplica feature can result in SolrCores being assigned new shards when using + legacyCloud=false and will also fail on a state check when taking over a core registration with a new + core. (Mark Miller, Hrishikesh Gadre, Patrick Dvorack) + +* SOLR-10184: Fix bin/solr so it can run properly on java9 (hossman, Uwe Schindler) + +* SOLR-9516: Admin UI (angular) now works with Kerberos, by excluding serving of /solr/libs/* through + SolrDispatchFilter. (Cassandra Targett, Amrit Sarkar via Ishan Chattopadhyaya) + +* SOLR-10302: Solr's zkcli scripts now able to find the metrics libraries, which it couldn't earlier (kiran, Ishan Chattopadhyaya) + +* SOLR-10283: Learning to Rank (LTR) SolrFeature to reject searches with missing efi (External Feature Information) used by fq. + (Christine Poerschke) + +* SOLR-10237: Poly-fields should work with subfields that have docValues=true (Tomás Fernández Löbbe, David Smiley) + +* SOLR-10218: The Schema API commands "add-field-type" and "replace-field-type" improperly specify SimilarityFactory params. + (Benjamin Deininger, Troy Mohl, Steve Rowe) + +* SOLR-10319: SolrCore "instanceDir" metric not visible in JMX. (ab) + +Optimizations +---------------------- + +* SOLR-9941: Clear the deletes lists at UpdateLog before replaying from log. This prevents redundantly pre-applying + DBQs, during the log replay, to every update in the log as if the DBQs were out of order. (hossman, Ishan Chattopadhyaya) + +* SOLR-9764: All filters that match all documents in the index now share the same memory (DocSet). + (Michael Sun, yonik) + +* SOLR-9584: Support Solr being proxied with another endpoint than default /solr, by using relative links + in AdminUI javascripts (Yun Jie Zhou via janhoy) + +* SOLR-10143: PointFields will create IndexOrDocValuesQuery when a field is both, indexed=true and docValues=true + (Tomás Fernández Löbbe) + +* SOLR-10273: The field with the longest value (if it exceeds 4K) is moved to be last in the Lucene Document in order + to benefit from stored field optimizations in Lucene that can avoid reading it when it's not needed. If the field is + multi-valued, they all move together to the end to retain order. (David Smiley) + +Other Changes +---------------------- +* SOLR-9980: Expose configVersion in core admin status (Jessica Cheng Mallet via Tomás Fernández Löbbe) + +* SOLR-9972: SpellCheckComponent collations and suggestions returned as a JSON object rather than a list + (Christine Poerschke in response to bug report from Ricky Oktavianus Lazuardy) + +* SOLR-9983: Fixing NullPointerException failure by TestManagedSchemaThreadSafety + adding check for Zookeeper session expiration (Steve Rowe, Mikhail Khludnev) + +* SOLR-10043: Reduce logging of pre-start log rotation (janhoy) + +* SOLR-10018: Increase the default hl.maxAnalyzedChars to 51200 for the Unified & Postings Highlighter so that all + highlighters now have this same default. (David Smiley) + +* SOLR-6246: Added tests to check that the changes in LUCENE-7564 and LUCENE-7670 + enable AnalyzingInfixSuggester and BlendedInfixSuggester to play nicely with core reload. + SolrSuggester.build() now throws SolrCoreState.CoreIsClosedException when interrupted + by a core reload/shutdown. (Steve Rowe) + +* SOLR-9800: Factor out FacetComponent.newSimpleFacets method. (Jonny Marks via Christine Poerschke) + +* SOLR-9914: SimpleFacets: refactor "contains" check into "SubstringBytesRefFilter" class. + (Jonny Marks, David Smiley, Christine Poerschke) + +* SOLR-10072: The test TestSelectiveWeightCreation appears to be unreliable. (Michael Nilsson via Mark Miller) + +* SOLR-10011: Refactor PointField & TrieField to now have a common base class, NumericFieldType. The + TrieField.TrieTypes and PointField.PointTypes are now consolidated to NumericFieldType.NumberType. This + refactoring also fixes a bug whereby PointFields were not using DocValues for range queries for + indexed=false, docValues=true fields. (Ishan Chattopadhyaya, Tomás Fernández Löbbe) + +* SOLR-9890: factor out ShardResultTransformerUtils.[un]marshSortValue methods + (Judith Silverman, Christine Poerschke) + +* SOLR-9966: Convert/migrate tests using EasyMock to Mockito (Cao Manh Dat, Uwe Schindler) + +* SOLR-10173: Make HttpShardHandlerFactory.getReplicaListTransformer more extensible. + (Ramsey Haddad via Christine Poerschke) + +* SOLR-9842: UpdateRequestProcessors have no way to guarantee the closing of resources used for a request. + (Mark Miller) + +* SOLR-9848: Lower solr.cloud.wait-for-updates-with-stale-state-pause back down from 7 seconds. + (Mark Miller) + +* SOLR-10020: Cannot reload a core if it fails initialization. (Mike Drob via Erick Erickson) + +* SOLR-9450: The docs/ folder in the binary distribution now contains a single index.html file linking + to the online documentation, reducing the size of the download (janhoy, Shawn Heisey, Uwe Schindler) + +* SOLR-7453: Remove replication & backup scripts in the solr/scripts directory of the checkout (Varun Thacker) + +* SOLR-10214: Remove unused HDFS BlockCache metrics and add storeFails, as well as adding total + counts for lookups, hits, and evictions. (yonik) + +* SOLR-10134: EmbeddedSolrServer responds on Schema API requests (Robert Alexandersson via Mikhail Khludnev) + +* SOLR-10219: re-enable HDFS tests under JDK9 (hossman, Uwe Schindler) + +* SOLR-10155: For numeric types facet.contains= and facet.prefix= are now rejected. + (Gus Heck, Christine Poerschke) + +* SOLR-10171: Add Constant Reduction Rules to Calcite Planner (Kevin Risden) + +* SOLR-10230: default TTL of PKIAuthenticationPlugin increased to 10secs (noble) + +* SOLR-10235: Fix DIH's TestJdbcDataSource to work with Java 9 and other Java runtimes that + do not use the same DriverManager implementation like Oracle's original one. The test now + uses a fully implemented Driver instance returning a mock connection. The test also works + correct now if other drivers were installed before test execution (e.g., through IDE). + (hossman, Uwe Schindler) + +* SOLR-8876: change morphline test config files to work around 'importCommands' bug when using java9 (hossman) + +* SOLR-10247: Support non-numeric metrics and a "compact" format of /admin/metrics output. (ab) + +* SOLR-9990: Add PointFields in example/default schemas (Tomás Fernández Löbbe) + +================== 6.4.2 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.14.v20161028 + +Detailed Change List +---------------------- + +Bug Fixes +---------------------- +* SOLR-10130, SOLR-10182: Serious performance degradation in Solr 6.4.1 due to the new metrics collection. + Default settings in solrconfig.xml /config/indexConfig/metrics have been changed to turn off + IndexWriter metrics collection. Directory level metrics collection has been completely removed until + a better design is found. (ab, ishan) + +* SOLR-10138: Transaction log replay can hit an NPE due to new Metrics code. (ab) + +* SOLR-10083: Fix instanceof check in ConstDoubleSource.equals (Pushkar Raste via Christine Poerschke) + +* SOLR-10190: Fix NPE in CloudSolrClient when reading stale alias (Janosch Woschitz via Tomás Fernández Löbbe) + +* SOLR-10192: Fix copy/paste in solr-ltr pom.xml template. (Christine Poerschke) + +================== 6.4.1 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.14.v20161028 + +Bug Fixes +---------------------- +* SOLR-9969: "Plugin/Stats" section of the UI doesn't display empty metric types (Tomás Fernández Löbbe) + +* SOLR-8491: solr.cmd SOLR_SSL_OPTS is overwritten (Sam Yi, Andy Hind, Marcel Berteler, Kevin Risden) + +* SOLR-10031: Validation of filename params in ReplicationHandler (Hrishikesh Gadre, janhoy) + +* SOLR-10054: Core swapping doesn't work with new metrics changes in place (ab) + +================== 6.4.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.14.v20161028 + +Detailed Change List +---------------------- + +Upgrade Notes +---------------------- + +* SOLR-9166: Export handler returns zero for numeric fields that are not in the original doc. One + consequence of this change is that you must be aware that some tuples will not have values if + there were none in the original document. + +* SOLR-8785: Metrics related classes in org.apache.solr.util.stats have been removed in favor of + the dropwizard metrics library. Any custom plugins using these classes should be changed to use + the equivalent classes from the metrics library. + As part of this, the following changes were made to the output of Overseer Status API: + * The "totalTime" metric has been removed because it is no longer supported + * The metrics "75thPctlRequestTime", "95thPctlRequestTime", "99thPctlRequestTime" + and "999thPctlRequestTime" in Overseer Status API have been renamed to "75thPcRequestTime", "95thPcRequestTime" + and so on for consistency with stats output in other parts of Solr. + * The metrics "avgRequestsPerMinute", "5minRateRequestsPerMinute" and "15minRateRequestsPerMinute" have been + replaced by corresponding per-second rates viz. "avgRequestsPerSecond", "5minRateRequestsPerSecond" + and "15minRateRequestsPerSecond" for consistency with stats output in other parts of Solr. + +* SOLR-9708: You are encouraged to try out the UnifiedHighlighter by setting hl.method=unified and report feedback. It + might become the default in 7.0. It's more efficient/faster than the other highlighters, especially compared to the + original Highlighter. That said, some options aren't supported yet. + It will get more features in time, especially with your input. See HighlightParams.java + for a listing of highlight parameters annotated with which highlighters use them. + hl.useFastVectorHighlighter is now considered deprecated in lieu of hl.method=fastVector. + +* SOLR-9712: maxWarmingSearchers now defaults to 1, and more importantly commits will now block if this + limit is exceeded instead of throwing an exception (a good thing). Consequently there is no longer a + risk in overlapping commits. Nonetheless users should continue to avoid excessive committing. Users are + advised to remove any pre-existing maxWarmingSearchers entries from their solrconfig.xml files. + +* SOLR-7466: complexphrase query parser now supports leading wildcards, beware of its' possible heaviness. + Users are encouraged to use ReversedWildcardFilter in index time analysis. + +New Features +---------------------- +* SOLR-9918: Add SkipExistingDocumentsProcessor that skips duplicate inserts and ignores updates to missing docs + (Tim Owen via koji) + +* SOLR-9293: Solrj client support for hierarchical clusters and other topics + marker. (Dawid Weiss) + +* SOLR-9681: FacetModule / JSON Facet API added the ability to add filters directly to + any facet command. The filters are applied after any domain change operations. + Example: { type:terms, field:category, filter:"user:yonik" } + (yonik) + +* SOLR-9442, SOLR-9787: Adds Array of Name Type Value (json.nl=arrntv) style to JSONResponseWriter. + (Jonny Marks, Christine Poerschke, hossman) + +* SOLR-8542: Adds Solr Learning to Rank (LTR) plugin for reranking results with machine learning models. + (Michael Nilsson, Diego Ceccarelli, Joshua Pantony, Jon Dorando, Naveen Santhapuri, Alessandro Benedetti, David Grohmann, Christine Poerschke) + +* SOLR-9055: Make collection backup/restore extensible. (Hrishikesh Gadre, Varun Thacker, Mark Miller) + +* SOLR-9682: JSON Facet API: added "param" query type to facet domain filter specification to obtain + filters via query parameters. (yonik) + +* SOLR-9038: Add a command-line tool to manage the snapshots functionality (Hrishikesh Gadre via yonik) + +* SOLR-9633: Limit memory consumed by FastLRUCache with a new 'maxRamMB' config parameter. + (yonik, Michael Sun, shalin) + +* SOLR-9666: SolrJ LukeResponse support dynamic fields (Fengtan via Kevin Risden) + +* SOLR-9077: Streaming expressions should support collection alias (Kevin Risden) + +* SOLR-9324: Support Secure Impersonation / Proxy User for solr authentication + (Gregory Chanan, Hrishikesh Gadre via yonik) + +* SOLR-9721: javabin Tuple parser for streaming and other end points (noble) + +* SOLR-9708: Added UnifiedSolrHighlighter, a highlighter adapter for Lucene's UnifiedHighlighter. The adapter is a + derivative of the PostingsSolrHighlighter, supporting mostly the same parameters with some differences. + Introduced "hl.method" parameter which can be set to original|fastVector|postings|unified to pick the highlighter at + runtime without the need to modify solrconfig from the default configuration. hl.useFastVectorHighlighter is now + considered deprecated in lieu of hl.method=fastVector. (Timothy Rodriguez, David Smiley) + +* SOLR-9728: Ability to specify Key Store type in solr.in.sh file for SSL (Michael Suzuki, Kevin Risden) + +* SOLR-5043: New solr.dns.prevent.reverse.lookup system property that can be used to prevent long core + (re)load delays on systems with missconfigured hostname/DNS (hossman) + +* SOLR-9844: FieldCache information fetched via the mbeans handler or seen via the UI now displays the total size used. + The individual cache entries in the response are now formatted better as well. (Varun Thacker) + +* SOLR-9513: Generic authentication plugins (GenericHadoopAuthPlugin and ConfigurableInternodeAuthHadoopPlugin) that delegate + all functionality to Hadoop authentication framework. (Hrishikesh Gadre via Ishan Chattopadhyaya) + +* SOLR-9860: Enable configuring invariantParams via HttpSolrClient.Builder (Hrishikesh Gadre, Ishan Chattopadhyaya) + +* SOLR-4735, SOLR-9921: Improve metrics reporting. This uses the dropwizard metric library, adding an internal + API for registering and reporting metrics from Solr components. Several new metrics and an improved JMX + reporter have been added (Alan Woodward, Jeff Wartes, Christine Poerschke, Kelvin Wong, shalin, ab) + +* SOLR-9788: Use instrumented jetty classes provided by the dropwizard metric library. (shalin) + +* SOLR-9805: Use metrics-jvm library to instrument jvm internals such as GC, memory usage and others. (shalin) + +* SOLR-9812, SOLR-9911, SOLR-9960: Added a new /admin/metrics API to return all metrics collected by Solr via API. + API supports four optional multi-valued parameters: + - 'group' (all,jvm,jetty,node,core), + - 'type' (all,counter,timer,gauge,histogram), + - 'prefix' that filters the returned metrics, + - 'registry' that selects one or more registries by prefix (eg. solr.jvm,solr.core.collection1) + - Example: http://localhost:8983/solr/admin/metrics?group=jvm,jetty&type=counter + - Example: http://localhost:8983/solr/admin/metrics?group=jvm&prefix=buffers,os + - Example: http://localhost:8983/solr/admin/metrics?registry=solr.node,solr.core&prefix=ADMIN + (shalin, ab) + +* SOLR-9884: Add version to segments handler output (Steven Bower via Erick Erickson) + +* SOLR-9725: Substitute properties into JdbcDataSource configuration ( Jamie Jackson, Yuri Sashevsky via Mikhail Khludnev) + +* SOLR-9877: SOLR-9923: SOLR-9948: Use instrumented http client and connection pool in HttpShardHandler and + UpdateShardHandler. The metrics are collected per query-less URL and method by default but it can be configured + to host/method and per-method as well. (shalin) + +* SOLR-9880: Add Ganglia, Graphite and SLF4J metrics reporters. (ab) + +* SOLR-9897: Add hl.requireFieldMatch toggle support when using the UnifiedHighlighter. Defaults to false like the + other highlighters that support this. (David Smiley) + +* SOLR-9905: Add NullStream to isolate the performance of the ExportWriter (Joel Bernstein) + +* SOLR-9891: Add mkroot command to bin/solr and bin/solr.cmd (Erick Erickson) + +* SOLR-9668,SOLR-7197: introduce cursorMark='true' in SolrEntityProcessor (Yegor Kozlov, Raveendra Yerraguntl via Mikhail Khludnev) + +* SOLR-9684: Add priority Streaming Expression (Joel Bernstein, David Smiley) + +* SOLR-9896: Instrument and collect metrics from query, update, core admin and core load thread pools. (shalin) + +* SOLR-9854: Collect metrics for index merges and index store IO (ab) + +* SOLR-8530: Add HavingStream to Streaming API and StreamingExpressions (Joel Bernstein) + +* SOLR-7466: Enable leading wildcard in complexphrase query parser, optimize it with ReversedWildcardFilterFactory + when it's provided (Mikhail Khludnev) + +* SOLR-9935: Add hl.fragsize support when using the UnifiedHighlighter to avoid snippets/Passages that are too small. + Defaults to 70. (David Smiley) + +* SOLR-9856: Collect metrics for shard replication and tlog replay on replicas (ab). + +* SOLR-9886: Add a 'enable' flag to caches to enable/disable them (Pushkar Raste, noble) + +* SOLR-9947: Clean up some SolrInfoMBean categories. Add an alternative hierarchical view in JMX + for SolrInfoMBeans, which uses similar conventions to SolrJmxReporter. (ab) + +Optimizations +---------------------- +* SOLR-9704: Facet Module / JSON Facet API: Optimize blockChildren facets that have + filters specified by using those filters as acceptDocs. (yonik) + +* SOLR-9726: Reduce number of lookupOrd calls made by the DocValuesFacets.getCounts method. + (Jonny Marks via Christine Poerschke) + +* SOLR-9772: Deriving distributed sort values (fieldSortValues) should reuse + comparator and only invalidate leafComparator. (John Call via yonik) + +* SOLR-9786: FieldType has a new getSetQuery() method that can take a set of terms + and create a more efficient query (such as TermsQuery). The solr query parser has been + changed to use this method when appropriate. The parser also knows when it is being + used to parse a filter and will create TermsQueries from large lists of normal terms + or numbers, resulting in a query that will execute faster. This also acts to avoid + BooleanQuery maximum clause limit. Query parsing itself has also been optimized, + resulting in less produced garbage and 5-7% better performance. + (yonik) + +* SOLR-9902: StandardDirectoryFactory should use Files API for it's move implementation. (Mark Miller, Mike Drob) + +Bug Fixes +---------------------- +* SOLR-9701: NPE in export handler when "fl" parameter is omitted. + (Erick Erickson) + +* SOLR-9433: SolrCore clean-up logic uses incorrect path to delete dataDir on failure to create a core. + (Evan Sayer, shalin) + +* SOLR-9360: Solr script not properly checking SOLR_PID + (Alessandro Benedetti via Erick Erickson) + +* SOLR-9716: RecoveryStrategy sends prep recovery command without setting read time out which can cause + replica recovery to hang indefinitely on network partitions. (Cao Manh Dat, shalin) + +* SOLR-9624: In Admin UI, do not attempt to highlight CSV output (Alexandre Rafalovitch) + +* SOLR-9005: In files example, add a guard condition to javascript URP script (Alexandre Rafalovitch) + +* SOLR-9519: JSON Facet API: don't stop at an empty facet bucket if any sub-facets still have a chance + of matching something due to filter exclusions (which can widen the domain again). + (Michael Sun, yonik) + +* SOLR-9740: A bug in macro expansion of multi-valued parameters caused non-expanded values + after the first expanded value in the same multi-valued parameter to be dropped. + (Erik Hatcher, yonik) + +* SOLR-9751: PreAnalyzedField can cause managed schema corruption. (Steve Rowe) + +* SOLR-9736: Solr resolves the collection name against the first available leader or first replica + of the first slice. This puts undue pressure on leader cores and likely on the wrong ones. This is + fixed to randomly pick a leader on updates or a replica core otherwise. (Cao Manh Dat via shalin) + +* SOLR-9284: The HDFS BlockDirectoryCache should not let it's keysToRelease or names maps grow indefinitely. + (Mark Miller, Michael Sun) + +* SOLR-9729: JDBCStream improvements (Kevin Risden) + +* SOLR-9626: new Admin UI now also highlights matched terms in the Analysis screen. (Alexandre Rafalovitch) + +* SOLR-9512: CloudSolrClient's cluster state cache can break direct updates to leaders (noble) + +* SOLR-5260: Facet search on a docvalue field in a multi shard collection (Trym Møller, Erick Erickson) + +* SOLR-9768: RecordingJsonParser produces incomplete json (Wojciech Stryszyk via ab) + +* SOLR-9616: Solr throws exception when expand=true on empty index (Timo Hund via Ishan Chattopadhyaya) + +* SOLR-9832: Schema modifications are not immediately visible on the coordinating node. (Steve Rowe) + +* SOLR-9834: A variety of spots in the code can create a collection zk node after the collection has been + removed. (Mark Miller) + +* SOLR-9707: Don't forward DeleteByQuery requests to down replicas. (Jessica Cheng Mallet via Varun Thacker) + +* SOLR-9823: CoreContainer incorrectly setting MDCLoggingContext for core (Jessica Cheng Mallet via Erick Erickson) + +* SOLR-1953: It may be possible for temporary files to accumulate until the Solr process is shut down. + (Karl Wright, Mark Miller) + +* SOLR-9847: Stop blocking further schema updates while waiting for a pending update to propagate to other replicas. + This reduces the likelihood of a (time-limited) distributed deadlock during concurrent schema updates. + (Mark Miller, Steve Rowe) + +* SOLR-9760: Windows script doesn't need write permission (Alex Crome by Mikhail Khludnev) + +* SOLR-9699,SOLR-4668: fix exception from core status in parallel with core reload (Mikhail Khludnev) + +* SOLR-9859: replication.properties cannot be updated after being written and neither replication.properties or + index.properties are durable in the face of a crash. (Pushkar Raste, Chris de Kok, Cao Manh Dat, Mark Miller) + +* SOLR-9901: Implement move in HdfsDirectoryFactory. (Mark Miller) + +* SOLR-9900: fix false positives on range queries with ReversedWildcardFilterFactory (Yonik Seeley via Mikhail Khludnev) + +* SOLR-9495: AIOBE with confusing message for incomplete sort spec in Streaming Expression (Gus Heck, Joel Bernstein) + +* SOLR-9154: Fix DirectSolrSpellChecker to work when added through the Config API. (Anshum Gupta) + +* SOLR-9919: random Streaming Expression is not registered in /stream or /graph handler (Joel Bernstein) + +* SOLR-7495: Support Facet.field on a non-DocValued, single-value, int field (Varun Thacker, Scott Stults) + +* SOLR-9917: JSON Facet API percentile function caused a NullPointerException in distributed mode when + there were no values in a bucket from a shard. (yonik) + +* SOLR-9931: JSON Facet API hll (hyper-log-log) function returned 0 for non-empty buckets with no field values + in local mode, but nothing for distributed mode. Both modes now return 0. (yonik) + +* SOLR-9503: NPE in Replica Placement Rules when using Overseer Role with other rules (Tim Owen via noble) + +* SOLR-9883: Example schemaless solr config files can lead to invalid tlog replays: when updates are buffered, + update processors ordered before DistributedUpdateProcessor, e.g. field normalization, are never run. (Steve Rowe) + +* SOLR-9644: SimpleMLTQParser and CloudMLTQParser did not handle field boosts properly + and CloudMLTQParser included extra strings from the field definitions in the query. + (Ere Maijala via Anshum Gupta) + +* SOLR-9954: Prevent against failure during failed snapshot cleanup from swallowing the actual cause + for the snapshot to fail. (thelabdude) + +Other Changes +---------------------- + +* SOLR-7539: Upgrade the clustering plugin to Carrot2 3.15.0. (Dawid Weiss) + +* SOLR-9621: Remove several Guava & Apache Commons calls in favor of java 8 alternatives. + (Michael Braun via David Smiley) + +* SOLR-9720: Refactor Responsewriters to remove dependencies on TupleStream, + Tuple, Explanation (noble) + +* SOLR-9717: Refactor '/export' to not hardcode the JSON output and to use an API (noble) + +* SOLR-9739: JavabinCodec implements PushWriter interface (noble) + +* SOLR-8332: Factor HttpShardHandler[Factory]'s url shuffling out into a ReplicaListTransformer class. + (Christine Poerschke, Noble Paul) + +* SOLR-9597: Add setReadOnly(String ...) to ConnectionImpl (Kevin Risden) + +* SOLR-9609: Change hard-coded keysize from 512 to 1024 (Jeremy Martini via Erick Erickson) + +* SOLR-8785: Use Dropwizard Metrics library for core metrics. The copied over code in + org.apache.solr.util.stats has been removed. (Jeff Wartes, Kelvin Wong, Christine Poerschke, shalin) + +* SOLR-9784: Refactor CloudSolrClient to eliminate direct dependency on ZK (noble) + +* SOLR-9801: Upgrade jetty to 9.3.14.v20161028 (shalin) + +* SOLR-9783: (Search|Top)Group[s]ShardResponseProcessor.process: turned sortWithinGroup null check into assert. + (Christine Poerschke) + +* SOLR-9660: in GroupingSpecification factor [group](sort|offset|limit) into [group](sortSpec) + (Judith Silverman, Christine Poerschke) + +* SOLR-9819: Upgrade commons-fileupload to 1.3.2, fixing a potential vulnerability CVE-2016-3092 (Anshum Gupta) + +* SOLR-9827: ConcurrentUpdateSolrClient creates a RemoteSolrException if the remote host responded with a non-ok + response (instead of a SolrException) and includes the remote error message as part of the exception message + (Tomás Fernández Löbbe) + +* SOLR-9846: OverseerAutoReplicaFailoverThread can take too long to stop and leak out of unit tests. (Mark Miller) + +* SOLR-8959: Refactored TestSegmentSorting out of TestMiniSolrCloudCluster (hossman) + +* SOLR-9874: Solr will reject CREATEALIAS requests if target collections don't exist (Tomás Fernández Löbbe) + +* SOLR-9878: fixing lazy logic for retrieving ReversedWildcardFilterFactory in SolrQueryParserBase (Mikhail Khludnev) + +* SOLR-9758: refactor preferLocalShards implementation (Christine Poerschke) + +* SOLR-9448: providing a test to workaround a differently named uniqueKey field (Mikhail Khludnev) + +* SOLR-9899: StandardDirectoryFactory should use optimizations for all FilterDirectorys not just NRTCachingDirectory. + (Mark Miller) + +* SOLR-9915: PeerSync alreadyInSync check is not backwards compatible and results in full replication during a rolling restart + (Tim Owen via noble) + +* SOLR-3990: Moves getIndexSize() from ReplicationHandler to SolrCore (Shawn Heisey) + +* SOLR-9944: Map the nodes function name to the GatherNodesStream (Joel Bernstein) + +* SOLR-9777: IndexFingerprinting should use getCombinedCoreAndDeletesKey() instead of getCoreCacheKey() for per-segment caching (Ishan Chattopadhyaya) + +* SOLR-9934: SolrTestCase.clearIndex has been improved to take advantage of low level test specific logic that + clears the index metadata more completely then a normal *:* DBQ can due to update versioning. (hossman) + +* SOLR-9893: Update Mockito to version 2.6.2 for Java 9 support. Disable all legacy EasyMock tests when running + on Java 9 until they were migrated to Mockito. (Uwe Schindler) + +================== 6.3.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.12.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.8.v20160314 + +Detailed Change List +---------------------- + +Upgrade Notes +---------------------- + +* If you use the JSON Facet API (json.facet) with method=stream, you must now set sort='index asc' to get the streaming + behavior; otherwise it won't stream. Reminder: "method" is a hint that doesn't change defaults of other parameters. + +* If you use the JSON Facet API (json.facet) to facet on a numeric field and if you use mincount=0 or if you set the + prefix, then you will now get an error as these options are incompatible with numeric faceting. + +* Solr's logging verbosity at the INFO level has been greatly reduced, and + you may need to update the log configs to use the DEBUG level to see all the + logging messages you used to see at INFO level before. + +* We are no longer backing up solr.log and solr_gc.log files in date-stamped copies forever. If you relied on + the solr_log_<date> or solr_gc_log_<date> being in the logs folder that will no longer be the case. + See SOLR-9570 for details. + +* The create/deleteCollection methods on MiniSolrCloudCluster have been + deprecated. Clients should instead use the CollectionAdminRequest API. In + addition, MiniSolrCloudCluster#uploadConfigDir(File, String) has been + deprecated in favour of #uploadConfigSet(Path, String) + +* The bin/solr.in.sh (bin/solr.in.cmd on Windows) is now completely commented by default. Previously, this wasn't so, + which had the effect of masking existing environment variables. + +New Features +---------------------- +* SOLR-5725: facet.method=enum can bypass exact counts calculation with facet.exists=true, it just returns 1 for + terms which exists in result docset. (Alexey Kozhemiakin, Sebastian Koziel, Radoslaw Zielinski via Mikhail Khludnev) + +* SOLR-9127: Excel workbook (.xlsx) response writer. use 'wt=xlsx' (Tony Moriarty, noble) + +* SOLR-9469: JettySolrRunner now has the option of restarting using a different + port (Alan Woodward) + +* SOLR-9319: DELETEREPLICA can accept a 'count' and remove appropriate replicas (Nitin Sharma, noble) + +* SOLR-8186: Reduce logging to logs/solr-<port>-console.log when not running in foreground mode + Show timestamp also in foreground log. Also removes some logging noise. (janhoy) + +* SOLR-8487: Adds CommitStream to support sending commits to a collection being updated. (Dennis Gove) + +* SOLR-9534: You can now set Solr's log level through environment variable SOLR_LOG_LEVEL + Also adds conveience arguments -q (quiet: WARN) and -v (verbose: DEBUG) to bin/solr (janhoy) + +* SOLR-9537: Support facet scoring with the scoreNodes expression (Joel Bernstein) + +* SOLR-9558: DIH TemplateTransformerto to support multivalued fields (Ted Sullivan via noble) + +* SOLR-9557: Every implicit requesthandler now has a default 'useParams' attribute (noble) + +* SOLR-9572: config API to show expanded useParams for request handlers inline (noble) + +* SOLR-9258: Optimizing, storing and deploying AI models with Streaming Expressions (Cao Manh Dat, Joel Bernstein) + +* SOLR-9205: Added method LukeResponse.getSchemaFlags() which returns field + information as an EnumSet (Fengtan, Alan Woodward) + +* SOLR-9520: Kerberos delegation support in SolrJ (Ishan Chattopadhyaya, noble) + +* SOLR-9146: Parallel SQL engine should support >, >=, <, <=, <>, != syntax (Timothy Potter, Joel Bernstein, Kevin Risden) + +* SOLR-9337: Add fetch Streaming Expression (Joel Bernstein) + +* SOLR-9103: Restore ability for users to add custom Streaming Expressions (Cao Manh Dat) + +* SOLR-9657: New TemplateUpdateProcessorFactory added (noble) + +* SOLR-9417: Allow daemons to terminate when they finish iterating a topic (Joel Bernstein) + +* SOLR-8370: Display configured Similarity in Schema-Browser, both global/default and per-field/field-type + (janhoy, Alexandre Rafalovitch) + +* SOLR-9326: Ability to create/delete/list snapshots at collection level. + (Hrishikesh Gadre via yonik) + +* SOLR-9662: New parameter -u <user:pass> in bin/post to pass basicauth credentials (janhoy) + +* SOLR-9654: Add "overrequest" parameter to JSON Facet API to control amount of overrequest + on a distributed terms facet. (yonik) + +* SOLR-2212: Add a factory class corresponding to Lucene's NoMergePolicy. (Lance Norskog, Cao Manh Dat via shalin) + +* SOLR-9670: Support SOLR_AUTHENTICATION_OPTS in solr.cmd (janhoy) + +* SOLR-9559: Add ExecutorStream to execute stored Streaming Expressions (Joel Bernstein) + +* SOLR-1085: Add support for MoreLikeThis queries and responses in SolrJ client. + (Maurice Jumelet, Bill Mitchell, Cao Manh Dat via shalin) + +Bug Fixes +---------------------- + +* SOLR-9310: PeerSync fails on a node restart due to IndexFingerPrint mismatch (Pushkar Raste, noble) + +* SOLR-9484: The modify collection API should wait for the modified properties to show up in the + cluster state. (Cao Manh Dat, shalin) + +* SOLR-9507: CoreContainer threads now correctly set their MDC logging values + (Alan Woodward) + +* SOLR-9522: Improve error handling in ZKPropertiesWriter (Varun Thacker) + +* SOLR-8080: bin/solr start script now exits with informative message if using wrong Java version (janhoy) + +* SOLR-9475: bin/install_solr_service.sh script got improved detection of Linux distro, especially within + virtualized/Docker environment through parsing of /etc/*-release files. Now also supports CentOS. (janhoy) + +* SOLR-9524: SolrIndexSearcher.getIndexFingerprint uses dubious synchronization (Mike Drob, noble) + +* SOLR-9542: Kerberos delegation tokens requires Jackson library (Ishan Chattopadhyaya via noble) + +* SOLR-9330: Fix AlreadyClosedException on admin/mbeans?stats=true (Mikhail Khludnev) + +* SOLR-9411: Better validation for Schema API add-field and add-dynamic-field (janhoy, Steve Rowe) + +* SOLR-9504: A replica with an empty index becomes the leader even when other more qualified replicas + are in line. (shalin) + +* SOLR-9554: Fix NullPointerException when cores are loaded in parallel and switch schema.xml to managed-scheme. + (Alan Woodward, Mikhail Khludnev) + +* SOLR-9556: OverseerAutoFailoverReplicaThread was not exiting on interrupt + (Alan Woodward) + +* SOLR-9563: Collection creation could fail if an ADDREPLICA subrequest arrived + at a node before its local state had updated with the new collection data + (Alan Woodward) + +* SOLR-9278: Index replication interactions with IndexWriter can cause deadlock. (Xunlong via Mark Miller) + +* SOLR-9604: Pooled SSL connections were not being re-used (Alan Woodward, + Mikhail Khludnev, hossman) + +* SOLR-9325: solr.log is now written to $SOLR_LOGS_DIR without changing log4j.properties (janhoy) + +* SOLR-9518: Kerberos Delegation Tokens don't work without a chrooted ZK (Ishan Chattopadhyaya,via noble) + +* SOLR-9687: Fixed Interval Facet count issue in cases of open/close intervals on the same values + (Andy Chillrud, Tomás Fernández Löbbe) + +* SOLR-9441: Solr collection backup on HDFS can only be manipulated by the Solr process owner. + (Hrishikesh Gadre via Mark Miller) + +* SOLR-9536: OldBackupDirectory timestamp field needs to be initialized to avoid NPE. + (Hrishikesh Gadre, hossman via Mark Miller) + +* SOLR-2039: Multivalued fields with dynamic names does not work properly with DIH. + (K A, ruslan.shv, Cao Manh Dat via shalin) + +* SOLR-4164: group.limit=-1 was not supported for grouping in distributed mode. + (Cao Manh Dat, Lance Norskog, Webster Homer, hossman, yonik) + +* SOLR-9692: blockUnknown property makes inter-node communication impossible (noble) + +* SOLR-2094: XPathEntityProcessor should reinitialize the XPathRecordReader instance if + the 'forEach' or 'xpath' attributes are templates & it is not a root entity (Cao Manh Dat, noble) + +* SOLR-9697: zk upconfig broken on windows (Xavier Jmlucjav via janhoy) + +Optimizations +---------------------- + +* SOLR-9374: Speed up Jmx MBean retrieval for FieldCache. (Tim Owen via shalin) + +* SOLR-9449: Example schemas do not index _version_ field anymore because the field + has DocValues enabled already. (shalin) + +* SOLR-9447: Do not clone SolrInputDocument if update processor chain does not contain custom processors. + (shalin) + +* SOLR-9452: JsonRecordReader should not deep copy document before handler.handle(). (noble, shalin) + +* SOLR-9142: JSON Facet API: new method=dvhash can be chosen for fields with high cardinality. (David Smiley) + +* SOLR-9446: Leader failure after creating a freshly replicated index can send nodes into recovery even if + index was not changed (Pushkar Raste, noble) + +* SOLR-9592: retrieving docValues as stored values was sped up by using the proper leaf + reader rather than ask for a global view. In extreme cases, this leads to a 100x speedup. + (Takahiro Ishikawa, yonik) + +* SOLR-9566: Don't put replicas into recovery when first creating a Collection + (Alan Woodward) + +* SOLR-9546: Eliminate unnecessary boxing/unboxing going on in SolrParams (Pushkar Raste, noble) + +* SOLR-9506: cache IndexFingerprint for each segment (Pushkar Raste, yonik, noble) + +* SOLR-7506: Roll over GC logs by default via bin/solr scripts (shalin, janhoy) + +Other Changes +---------------------- + +* SOLR-9412: Add failOnMissingParams option to MacroExpander, add TestMacroExpander class. + (Jon Dorando, Christine Poerschke) + +* SOLR-9406: SolrSuggester should selectively register close hook (Gethin James, Joel Bernstein) + +* SOLR-8961: Add a test module for solr-test-framework (Alan Woodward) + +* SOLR-9474: MiniSolrCloudCluster will not reuse ports by default when + restarting its JettySolrRunners (Alan Woodward) + +* SOLR-9498: Remove HDFS properties from DIH solrconfig.xml, as started in SOLR-6943 (Alexandre Rafalovitch) + +* SOLR-9365: Reduce noise in solr logs during graceful shutdown. (Cao Manh Dat via shalin) + +* SOLR-9451: Make clusterstatus command logging less verbose. (Varun Thacker) + +* SOLR-9502: ResponseWriters should natively support MapSerializable (noble) + +* SOLR-9538: Relocate (BinaryResponse|JSON|Smile)Writer tests to org.apache.solr.response + which is the package of the classes they test. (Jonny Marks via Christine Poerschke) + +* SOLR-9508: Install script install_solr_service.sh now checks existence of tools. + New option -n to avoid starting service after installation (janhoy) + +* SOLR-7826: Refuse "bin/solr create" if run as root, unless -force is specified (janhoy, Binoy Dalal) + +* SOLR-6871: Updated the quickstart tutorial to cover the 6.2.0 release, and added ant target + "generate-website-quickstart" to convert the bundled version of the tutorial into one suitable + for the website. + +* SOLR-5563: Move lots of SolrCloud logging from 'info' to 'debug' (janhoy, Alan + Woodward) + +* SOLR-9544: Allow ObjectReleaseTracker more time to check for asynchronously + closing resources (Alan Woodward) + +* SOLR-6677: Reduced logging during startup and shutdown, moved more logs to DEBUG level + (janhoy, Shawn Heisey, Alan Woodward) + +* SOLR-6090: Remove unreachable printLayout usage in cloud tests. (Cao Manh Dat via shalin) + +* SOLR-9551: Add JSONWriter constructor variant, JSONWriterTest.testConstantsUnchanged test. + (Jonny Marks, Christine Poerschke) + +* SOLR-9500: Add a LogLevel annotation to set log levels on specific tests (Alan + Woodward) + +* SOLR-9548: The beginning of solr.log now starts with a more informative welcome message (janhoy) + +* SOLR-9547: Do not allow bin/solr start as root user, unless -force param specified (janhoy) + +* SOLR-9567: Make ReRankQParserPlugin's private ReRankCollector a public class of its own. (Christine Poerschke) + +* SOLR-7436: Solr stops printing stacktraces in log and output (janhoy, hossman, Markus Jelsma) + +* SOLR-9576: Make FieldAnalysisRequestHandler, DocumentAnalysisRequestHandler & DumpRequestHandler + implicit (noble) + +* SOLR-9574: Factor out AbstractReRankQuery from ReRankQParserPlugin's private ReRankQuery. (Christine Poerschke) + +* SOLR-5041: Add a test to make sure that a leader always recovers from log on startup. (Cao Manh Dat, shalin) + +* SOLR-9588: Remove Guava dependency from SolrJ (Ishan Chattopadhyaya, noble) + +* SOLR-8140: Remove mentions of unimplemented admin-extra from the new Admin UI (Alexandre Rafalovitch) + +* SOLR-9589: Remove jackson dependency from SolrJ (Ishan Chattopadhyaya, noble) + +* SOLR-8385: Narrow StreamFactory.withFunctionName clazz parameter to prevent misconfiguration (Jason Gerlowski, Kevin Risden) + +* SOLR-8969: SQLHandler causes NPE in non-cloud mode (Markus Jelsma, Kevin Risden) + +* SOLR-9610: New AssertTool in SolrCLI for easier cross platform assertions from command line (janhoy) + +* SOLR-9680: Better error messages in SolrCLI when authentication required (janhoy) + +* SOLR-9639: Test only fix. Prevent CDCR tests from removing collection during recovery that used to blow up jvm (Mikhail Khludnev) + +* SOLR-9625: Add HelloWorldSolrCloudTestCase class (Christine Poerschke, Alan Woodward, Alexandre Rafalovitch) + +* SOLR-9642: Refactor the core level snapshot cleanup mechanism to rely on Lucene (Hrishikesh Gadre via yonik) + +* SOLR-9627: Add QParser.getSortSpec, deprecate misleadingly named QParser.getSort (Judith Silverman, Christine Poerschke) + +* SOLR-9632: Add MiniSolrCloudCluster#deleteAllCollections() method (Alan Woodward) + +* SOLR-9634: Deprecate collection methods on MiniSolrCloudCluster (Alan Woodward) + +* SOLR-7850: Moved defaults within bin/solr.in.sh (and bin/solr.in.cmd on Windows) to bin/solr (and bin/solr.cmd) + such that the default state of these files is to set nothing. This makes Solr work better with Docker. (David Smiley) + +* SOLR-9570: Various log tidying now happens at Solr startup: + Old solr_log_<date> and solr_gc_log_<date> files are removed, avoiding disks to fill up, + solr.log.X files are rotated, preserving solr.log from last run in solr.log.1, solr.log.1 => solr.log.2 etc + solr-*-console.log files are moved into $SOLR_LOGS_DIR/archived/ instead of being overwritten + Last JVM garbage collection log solr_gc.log is moved into $SOLR_LOGS_DIR/archived/ + (janhoy) + +* SOLR-4531: Add tests to ensure that recovery does not fail on corrupted tlogs. + (Simon Scofield, Cao Manh Dat via shalin) + +* SOLR-5245: Add a test to ensure that election contexts are keyed off both collection name and coreNodeName + so that killing a shard in one collection does not result in leader election in a different collection. + See SOLR-5243 for the related bug. (Cao Manh Dat via shalin) + +* SOLR-9533: Reload core config when a core is reloaded (Gethin James, Joel Bernstein) + +* SOLR-9371: Fix bin/solr calculations for start/stop wait time and RMI_PORT. + (Shawn Heisey via Erick Erickson) + +================== 6.2.1 ================== + +Bug Fixes +---------------------- + +* SOLR-9494: Use of {!collapse} sometimes doesn't correctly return true for Collector.needsScores(), especially when the + query was cached. This can cause an exception when 'q' is a SpanQuery or potentially others. (David Smiley) + +* SOLR-6744: fl renaming / alias of uniqueKey field generates null pointer exception in SolrCloud configuration + (Mike Drob via Tomás Fernández Löbbe) + +* SOLR-9445: Admin requests are retried by CloudSolrClient and LBHttpSolrClient on failure. (shalin) + +* SOLR-9439: Shard split clean up logic for older failed splits is faulty. The delete shard API + has also been made more resilient against failures resulting from non-existent cores. (shalin) + +* SOLR-9430: Fix locale lookup in DIH <propertyWriter/> to use BCP47 language tags + to be consistent with other places in Solr. Language names still work for backwards + compatibility. (Uwe Schindler, Boris Steiner) + +* SOLR-9389: HDFS Transaction logs stay open for writes which leaks Xceivers. (Tim Owen via Mark Miller) + +* SOLR-9188: blockUnknown property makes inter-node communication impossible (noble) + +* SOLR-9455: Deleting a sub-shard in recovery state can mark parent shard as inactive. (shalin) + +* SOLR-9461: DELETENODE, REPLACENODE should pass down the 'async' param to subcommands (shalin, noble) + +* SOLR-9444: Fix path usage for cloud backup/restore. (Hrishikesh Gadre, Uwe Schindler, Varun Thacker) + +* SOLR-9381: Snitch for freedisk uses '/' instead of 'coreRootDirectory' (Tim Owen, noble) + +* SOLR-9488: Shard split can fail to write commit data on shutdown/restart causing replicas to recover + without replicating the index. This can cause data loss. (shalin) + +* SOLR-9490: Fixed bugs in BoolField that caused it to erroneously return "false" for all docs depending + on usage (Colvin Cowie, Dan Fox, hossman) + +* SOLR-9438: Shard split can be marked successful and sub-shard states switched to 'active' even when + one or more sub-shards replicas do not recover due to the leader crashing or restarting between the time + the replicas are created and before they can recover. This can cause data loss. (shalin) + +* SOLR-9408: Fix TreeMergeOutputFormat to add timestamp metadata to a commit. SolrCloud replication relies on this. + (Jessica Cheng Mallet via Varun Thacker) + +Other Changes +---------------------- + +* SOLR-7362: Fix TestReqParamsAPI test failures (noble, Varun Thacker) + +================== 6.2.0 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.12.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.8.v20160314 + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-9187: Support dates and booleans in /export handler, support boolean DocValues fields (Erick Erickson) + +* SOLR-8048: bin/solr script should support basic auth credentials provided in solr.in.sh (noble) + +* SOLR-7374: Core level Backup/Restore now supports specifying the directory implementation to use + via the "repository" parameter. (Hrishikesh Gadre, Varun Thacker, Mark Miller) + +* SOLR-9216: Support collection.configName in MODIFYCOLLECTION request (Keith Laban, noble) + +* SOLR-9251: Support for a new tag 'role' in replica placement rules (noble) + +* SOLR-9194: Enhance the bin/solr script to perform file operations to/from Zookeeper (Erick Erickson, janhoy) + +* SOLR-9242: Collection Backup/Restore now supports specifying the directory implementation to use + via the "repository" parameter. (Hrishikesh Gadre, Varun Thacker) + +* SOLR-9193: Add scoreNodes Streaming Expression (Joel Bernstein) + +* SOLR-9243: Add terms.list parameter to the TermsComponent to fetch the docFreq for a list of terms + (Joel Bernstein) + +* SOLR-9090: Add directUpdatesToLeadersOnly flag to solrj CloudSolrClient. + (Marvin Justice, Christine Poerschke) + +* SOLR-9270: Allow spatialContextFactory to be simply "JTS". And if any spatial params include the old + Spatial4j package "com.spatial4j.core" it is rewritten to "org.locationtech.spatial4j" with a warning. + (David Smiley) + +* SOLR-9240: Support parallel ETL with the topic expression (Joel Bernstein) + +* SOLR-9275: XML QueryParser support (defType=xmlparser) now extensible via configuration. + (Christine Poerschke) + +* SOLR-9200: Add Delegation Token Support to Solr. + (Gregory Chanan) + +* SOLR-9038: Solr core snapshots: The current commit can be snapshotted which retains the commit and associates it with + a name. The core admin API can create snapshots, list them, and delete them. Snapshot names can be referenced in + doing a core backup, and in replication. Snapshot metadata is stored in a new snapshot_metadata/ dir. + (Hrishikesh Gadre via David Smiley) + +* SOLR-9279: New boolean comparison function queries comparing numeric arguments: gt, gte, lt, lte, eq + (Doug Turnbull, David Smiley) + +* SOLR-9252: Feature selection and logistic regression on text (Cao Manh Dat, Joel Bernstein) + +* SOLR-6465: CDCR: fall back to whole-index replication when tlogs are insufficient. + (Noble Paul, Renaud Delbru, shalin) + +* SOLR-9320: A REPLACENODE command to decommission an existing node with another new node + (noble, Nitin Sharma, Varun Thacker) + +* SOLR-9318: A DELETENODE command to delete all replicas in that node (noble, Nitin Sharma, Varun Thacker) + +Bug Fixes +---------------------- + +* SOLR-9191: OverseerTaskQueue.peekTopN() fatally flawed (Scott Blum, Noble Paul) + +* SOLR-9199: ZkController#publishAndWaitForDownStates logic is inefficient (Hrishikesh Gadre) + +* SOLR-9161: Change SolrPluginUtils.invokeSetters implementation to accommodate setter variants. + (Christine Poerschke, Steve Rowe, Uwe Schindler) + +* SOLR-9234: srcField parameter works only when all fields are captured in the /update/json/docs + endpoint (noble) + +* SOLR-8546: SOLR-8546: TestLazyCores is failing a lot on the Jenkins cluster. (Erick Erickson) + +* SOLR-9237: DefaultSolrHighlighter.doHighlightingByFastVectorHighlighter can't be overidden (janhoy) + +* SOLR-8626: 404 error when clicking nodes in cloud graph view in angular UI. (janhoy, Trey Grainger via shalin) + +* SOLR-9254: GraphTermsQueryQParserPlugin throws NPE when field being searched is not present in segment + (Joel Bernstein) + +* SOLR-8657: Fix SolrRequestInfo error logs if QuerySenderListener is being used (Pascal Chollet, + Tomás Fernández Löbbe) + +* SOLR-8777: Duplicate Solr process can cripple a running process. (Jessica Cheng Mallet, Scott Blum, shalin) + +* SOLR-9246: If the JDBCStream sees an unknown column type it will now throw a detailed exception. (Dennis Gove) + +* SOLR-9181: Fix some races in CollectionStateWatcher API (Alan Woodward, Scott + Blum) + +* SOLR-9235: Fixed NPE when using non-numeric range query in deleteByQuery (hossman) + +* SOLR-9088: Fixed TestManagedSchemaAPI failures which exposed race conditions in the schema API ( Varun Thacker, noble) + +* SOLR-9207: PeerSync recovery failes if number of updates requested is high. A new useRangeVersions config option + is introduced (defaults to true) to send version ranges instead of individual versions for peer sync. + (Pushkar Raste, shalin) + +* SOLR-8858: SolrIndexSearcher#doc() completely ignores field filters unless lazy field loading is enabled. + (Caleb Rackliffe, David Smiley, shalin) + +* SOLR-9236: AutoAddReplicas will append an extra /tlog to the update log location on replica failover. + (Eungsop Yoo, Mark Miller) + +* SOLR-9291: ZkSolrResourceLoader should not retry fetching resources if the server has been shutdown. + (shalin) + +* SOLR-9287: Including 'score' in the 'fl' param when doing an RTG no longer causes an NPE + (hossman, Ishan Chattopadhyaya) + +* SOLR-7280: In cloud-mode sort the cores smartly before loading & limit threads to improve cluster stability + (noble, Erick Erickson, shalin) + +* SOLR-9285: Fixed AIOOBE when using ValueSourceAugmenter in single node RTG (hossman) + +* SOLR-9288: Fix [docid] transformer to return -1 when used in RTG with uncommitted doc (hossman) + +* SOLR-9309: Fix SolrCloud RTG response structure when multi ids requested but only 1 found (hossman) + +* SOLR-9334: CloudSolrClient.collectionStateCache is unbounded (noble) + +* SOLR-9339: NPE in CloudSolrClient when the response is null (noble) + +* SOLR-8596: Web UI doesn't correctly generate queries which include local parameters (Alexandre Rafalovitch, janhoy) + +* SOLR-8645: managed-schema is now syntax highlighted in cloud->Tree view (Alexandre Rafalovitch via janhoy) + +* SOLR-8379: UI Cloud->Tree view now shows .txt files correctly (Alexandre Rafalovitch via janhoy) + +* SOLR-9003: New Admin UI's Dataimport screen now correctly displays DIH Debug output (Alexandre Rafalovitch) + +* SOLR-9308: Fix distributed RTG to forward request params, fixes fq and non-default fl params (hossman) + +* SOLR-9179: NPE in IndexSchema using IBM JDK (noble, Colvin Cowie) + +* SOLR-9397: Config API does not support adding caches (noble) + +* SOLR-9405: ConcurrentModificationException in ZkStateReader.getStateWatchers. + (Alan Woodward, Edward Ribeiro, shalin) + +* SOLR-9232: Admin UI now fully implements Swap Cores interface (Alexandre Rafalovitch) + +* SOLR-8715: Admin UI's Schema screen now works for fields with stored=false and some content indexed (Alexandre Rafalovitch) + +* SOLR-8911: In Admin UI, enable scrolling for overflowing Versions and JVM property values (Alexandre Rafalovitch) + +* SOLR-9002: Admin UI now correctly displays json and text files in the collection/Files screen (Upayavira, Alexandre Rafalovitch) + +* SOLR-8993: Admin UI now correctly supports multiple DIH handler end-points (Upayavira, Alexandre Rafalovitch) + +* SOLR-9032: Admin UI now correctly implements Create Alias command (Upayavira, Alexandre Rafalovitch) + +* SOLR-9391: LBHttpSolrClient.request now correctly returns Rsp.server when + previously skipped servers were successfully tried. (Christine Poerschke) + + +Optimizations +---------------------- + +* SOLR-9219: Make hdfs blockcache read buffer sizes configurable and improve cache concurrency. (Mark Miller) + +* SOLR-9264: Optimize ZkController.publishAndWaitForDownStates to not read all collection states and + watch relevant collections instead. (Hrishikesh Gadre, shalin) + +* SOLR-9335: Solr cache/search/update stats counters now use LongAdder which are supposed to have higher throughput + under high contention. (Varun Thacker) + +* SOLR-9350: JSON Facets: method="stream" will no longer always uses & populates the filter cache, likely + flushing it. 'cacheDf' can be configured to set a doc frequency threshold, now defaulting to 1/16th doc count. + Using -1 Disables use of the cache. (David Smiley, yonik) + +Other Changes +---------------------- + +* SOLR-9195: Remove unnecessary allocation and null check in UpdateRequestProcessorChain's + getReqProcessors method. (Christine Poerschke) + +* SOLR-8981: Upgraded Extraction module to Apache Tika 1.13. + (Tim Allison, Lewis John McGibbney via Uwe Schindler) + +* SOLR-9076: Update to Hadoop 2.7.2 + (Mark Miller, Gregory Chanan) + +* SOLR-8787: TestAuthenticationFramework should not extend TestMiniSolrCloudCluster. (Trey Cahill via shalin) + +* SOLR-9180: More comprehensive tests of psuedo-fields for RTG and SolrCloud requests (hossman) + +* SOLR-7930: Comment out trappy references to example docs in elevate.xml files (Erick Erickson) + +* SOLR-9277: Clean up some more remnants of supporting old and new style solr.xml in tests (Erick Erickson) + +* SOLR-9163: Sync up basic_configs and data_driven_schema_configs, removing almost all differences + except what is required for schemaless. (yonik) + +* SOLR-9340: Change ZooKeeper disconnect and session expiry related logging from INFO to WARN to + make debugging easier (Varun Thacker) + +* SOLR-9358: [AngularUI] In Cloud->Tree file view area, collapse metadata by default (janhoy) + +* SOLR-9256: asserting hasNext() contract in JdbcDataSource in DataImportHandler (Kristine Jetzke via Mikhai Khludnev) + +* SOLR-9209: extracting JdbcDataSource.createResultSetIterator() for extension (Kristine Jetzke via Mikhai Khludnev) + +* SOLR-9353: Factor out ReRankQParserPlugin.ReRankQueryRescorer private class. (Christine Poerschke) + +* SOLR-9392: Fixed CDCR Test failures which were due to leaked resources. (shalin) + +* SOLR-9385: Add QParser.getParser(String,SolrQueryRequest) variant. (Christine Poerschke) + +* SOLR-9367: Improved TestInjection's randomization logic to use LuceneTestCase.random() (hossman) + +* SOLR-9331: Remove ReRankQuery's length constructor argument and member. (Christine Poerschke) + +* SOLR-9092: For the delete replica command we attempt to send the core admin delete request only + if that node is actually up. (Jessica Cheng Mallet, Varun Thacker) + +* SOLR-9410: Make ReRankQParserPlugin's private ReRankWeight a public class of its own. (Christine Poerschke) + +* SOLR-9404: Refactor move/renames in JSON FacetProcessor and FacetFieldProcessor. (David Smiley) + +* SOLR-9421: Refactored out OverseerCollectionMessageHandler to smaller classes (noble) + +* SOLR-8643: BlockJoinFacetComponent is substituted by BlockJoinFacetDocSetComponent. It doesn't need to change solrconfig.xml (Mikhail Khludnev) + +* SOLR-8644: Test asserts that block join facets work with parent level fq exclusions. (Dr. Oleg Savrasov via Mikhail Khludnev) + +================== 6.1.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.7 +Carrot2 3.12.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.8.v20160314 + +Upgrading from Solr any prior release +---------------------- + +* If you use historical dates, specifically on or before the year 1582, you should re-index. + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-8782: Add asynchronous sugar methods to the SolrJ Collections API. You + can now call .processAsync() to run a method asynchronously, or + .processAndWait() to wait for a call to finish without holding HTTP + collections open. (Alan Woodward) + +* SOLR-8765: Enforce required parameters at query construction time in the SolrJ + Collections API, add static factory methods, and deprecate old setter methods. + (Alan Woodward, Jason Gerlowski) + +* SOLR-8842: authorization APIs do not use name as an identifier for a permission + for update, delete commands and 'before' attribute (noble) + +* SOLR-8814: Support GeoJSON response writer and general spatial formatting. Adding + &wt=geojson&geojson.field=<your geometry field> + Will return a FeatureCollection for each SolrDocumentList and a Feature with the + requested geometry for each SolrDocument. The requested geometry field needs + to either extend AbstractSpatialFieldType or store a GeoJSON string. This also adds + a [geo] DocumentTransformer that can return the Shape in a variety of formats: + &fl=[geo f=<your geometry field> w=(GeoJSON|WKT|POLY)] + The default format is GeoJSON. For information on the supported formats, see: + https://github.com/locationtech/spatial4j/blob/master/FORMATS.md + To return the FeatureCollection as the root element, add '&omitHeader=true" (ryan) + +* SOLR-8859: Spatial fields like RPT can now be configured to use Spatial4j registered shape formats + e.g. via format="GeoJSON". (ryan, David Smiley) + +* SOLR-445: new TolerantUpdateProcessorFactory to support skipping update commands that cause + failures when sending multiple updates in a single request. + (Erick Erickson, Tomás Fernández Löbbe, Anshum Gupta, hossman) + +* SOLR-8890: New static method in DistributedUpdateProcessorFactory to allow UpdateProcessorFactories + to indicate request params that should be forwarded when DUP distributes updates. (hossman) + +* SOLR-8888: Add shortestPath Streaming Expression. This expression performs a breadth first + graph traversal to find the shortest path(s) in an unweighted, directed graph stored in a + SolrCloud collection. (Joel Bernstein) + +* SOLR-8938: Add optional --excluderegex argument to ZkCLI. (Christine Poerschke) + +* SOLR-8976: Add SolrJ support for REBALANCELEADERS Collections API (Anshum Gupta) + +* SOLR-8962: Add sort Streaming Expression. The expression takes a single input stream and a + comparator and outputs tuples in stable order of the comparator. (Dennis Gove) + +* SOLR-8349: Allow sharing of large in memory data structures across cores (Gus Heck, noble) + +* SOLR-9009: Adds ability to get an Explanation of a Streaming Expression (Dennis Gove) + +* SOLR-8918: Adds Streaming to the admin page under the collections section. Includes + ability to see graphically the expression explanation (Dennis Gove) + +* SOLR-8913: When using a shared filesystem we should store data dir and tlog dir locations in + the cluster state. (Mark Miller) + +* SOLR-8809: Implement Connection.prepareStatement (Kevin Risden) + +* SOLR-9020: Implement StatementImpl/ResultSetImpl get/set fetch* methods and proper errors for traversal methods (Kevin Risden) + +* SOLR-9041: 'core-admin-read' and 'core-admin-edit' are well known permissions (noble) + +* SOLR-8986: Add Random Streaming Expression (Joel Bernstein) + +* SOLR-8925: Add gatherNodes Streaming Expression to support breadth first traversals (Joel Bernstein) + +* SOLR-9027: Add GraphTermsQuery to limit traversal on high frequency nodes (Joel Bernstein, David Smiley) + +* SOLR-5750: Add /admin/collections?action=BACKUP and RESTORE assuming access to a shared file system. + (Varun Thacker, David Smiley) + +* SOLR-9049: RuleBasedAuthorizationPlugin supports regex in param values eg: "command" : "REGEX:(i?)create" (noble) + +* SOLR-8972: Add GraphHandler and GraphMLResponseWriter to support graph visualizations (Joel Bernstein) + +* SOLR-9026: Extend facet telemetry support to legacy (non-json) facets under "debug/facet-debug" in + the response. (Michael Sun, yonik) + +* SOLR-7117: Provide an option to limit the maximum number of cores that can be created on a node by the + Auto Add Replica feature. For this you can set a "maxCoresPerNode" property via the Cluster Property API + (Varun Thacker, Mark Miller) + +* SOLR-8208: [subquery] document transformer executes separate requests per result document. (Cao Manh Dat via Mikhail Khludnev) + +* SOLR-8323, SOLR-9113: Add CollectionStateWatcher API (Alan Woodward, Scott Blum) + +* SOLR-8988: Adds query option facet.distrib.mco which when set to true allows the use of facet.mincount=1 in cloud mode. + (Keith Laban, Dennis Gove) + +* SOLR-8583: Apply highlighting to hl.alternateField by default for Default and FastVectorHighlighter. + Turn off with hl.highlightAlternate=false (janhoy, David Smiley) + +* SOLR-7123: '/update/json/docs' path supports nested documents (noble) + +* SOLR-8610: Resolve variables in encryptKeyFile of DIH's JdbcDataSource (Kristine Jetzke via Mikhail Khludnev) + +* SOLR-7739: Add a new ClassificationUpdateProcessorFactory. (Alessandro Benedetti via Tommaso Teofili) + +Bug Fixes +---------------------- + +* SOLR-8855: The HDFS BlockDirectory should not clean up it's cache on shutdown. (Mark Miller) + +* SOLR-8948: OverseerTaskQueue.containsTaskWithRequestId encounters json parse error if a + SolrResponse node is in the overseer queue. (Jessica Cheng Mallet via shalin) + +* SOLR-7729: ConcurrentUpdateSolrClient ignores the collection parameter in some methods. + (Nicolas Gavalda, Jorge Luis Betancourt Gonzalez via Mark Miller) + +* SOLR-8662: SchemaManager waits correctly for replicas to be notified of a new change. + (sarowe, Noble Paul, Varun Thacker) + +* SOLR-8983: Cleanup clusterstate and replicas for a failed create collection request + (Varun Thacker, Anshum Gupta) + +* SOLR-9029: fix rare ZkStateReader visibility race during collection state format update (Scott Blum, hossman) + +* SOLR-9046: Fix solr.cmd that wrongly assumes Jetty will always listen on 0.0.0.0. + (Bram Van Dam, Uwe Schindler) + +* SOLR-9064: Adds an explanation of the incoming stream to an UpdateStream's explanation (Dennis Gove) + +* SOLR-9128: Fix error handling issues in Streaming classes (Joel Bernstein) + +* SOLR-9151: Fix SolrCLI so that bin/solr -e cloud example can be run from any CWD (janhoy) + +* SOLR-9141: Fix ClassCastException when using the /sql handler count() function with + single-shard collections (Minoru Osuka via James Dyer) + +* SOLR-9165: Spellcheck does not return collations if "maxCollationTries" is used with "cursorMark". + (James Dyer) + +* SOLR-8940: Fix group.sort option (hossman) + +* SOLR-8612: closing JDBC Statement on failures in DataImportHandler (DIH) (Kristine Jetzke via Mikhail Khludnev) + +* SOLR-8676: keep LOG4J_CONFIG in solr.cmd (Kristine Jetzke via Mikhail Khludnev) + +* SOLR-9198: config APIs unable to add multiple values with same name (noble) + +* SOLR-8812: edismax: turn off mm processing if no explicit mm spec is provided + and there are explicit operators (except for AND) - addresses problems caused by SOLR-2649. + (Greg Pendlebury, Jan Høydahl, Erick Erickson, Steve Rowe) + +* SOLR-9176: facet method ENUM was sometimes unnecessarily being rewritten to + FCS, causing slowdowns (Alessandro Benedetti, Jesse McLaughlin, Alan Woodward) + +Optimizations +---------------------- +* SOLR-8722: Don't force a full ZkStateReader refresh on every Overseer operation. + (Scott Blum via shalin) + +* SOLR-8745: Deprecate costly ZkStateReader.updateClusterState(), replace with a narrow + forceUpdateCollection(collection) (Scott Blum via shalin) + +* SOLR-8856: Do not cache merge or 'read once' contexts in the hdfs block cache. (Mark Miller, Mike Drob) + +* SOLR-8922: Optimize filter creation (DocSetCollector) to minimize the amount of garbage + produced. This resulted in up to 3x throughput when small filter creation was the bottleneck, + as well as orders of magnitude less garbage. (Jeff Wartes, yonik) + +* SOLR-8937: bin/post (SimplePostTool) now streams the standard input instead of buffering fully. + (David Smiley) + +* SOLR-8973: Zookeeper frenzy when a core is first created. (Janmejay Singh, Scott Blum, shalin) + +* SOLR-9014: Deprecate and reduce usage of ClusterState methods which may make calls to ZK via + the lazy collection reference. (Scott Blum, shalin) + +* SOLR-9106: Cluster properties are now cached on ZkStateReader. (Alan Woodward) + +* SOLR-9147: Upgrade commons-io to 2.5, avoid expensive array resizing in EmbeddedSolrServer (Mikhail Khludnev) + +* SOLR-8744: Overseer operations performed with fine grained mutual exclusion (noble, Scott Blum) + +* SOLR-9204: Improve performance of getting directory size with hdfs. (Mark Miller) + +Other Changes +---------------------- +* SOLR-8860: Remove back-compat handling of router format made in SOLR-4221 in 4.5.0. (shalin) + +* SOLR-8866: UpdateLog will now throw an exception if it doesn't know how to serialize a value. + (David Smiley) + +* SOLR-8842: security rules made more foolproof by asking the requesthandler about the well known + permission name.
The APIs are also modified to ue 'index' as the unique identifier instead of name. + Name is an optional attribute
now and only to be used when specifying well-known permissions (noble) + +* SOLR-5616: Simplifies grouping code to use ResponseBuilder.needDocList() to determine if it needs to + generate a doc list for grouped results. (Steven Bower, Keith Laban, Dennis Gove) + +* SOLR-8869: Optionally disable printing field cache entries in SolrFieldCacheMBean (Gregory Chanan) + +* SOLR-8892: Allow SolrInfoMBeans to return different statistics for /jmx vs web ui calls. + (Gregory Chanan, Mark Miller) + +* SOLR-8097: Implement builder pattern design for constructing SolrJ clients and also deprecate direct construction + of client objects. (Jason Gerlowski, Shawn Heisey, Anshum Gupta) + +* SOLR-9015: Adds SelectStream as a default function in the StreamHandler (Dennis Gove) + +* SOLR-8929: Add an idea module for solr/server to enable launching start.jar (Scott Blum, Steve Rowe) + +* SOLR-8933: Solr should not close container streams. (Mike Drob, Uwe Schindler, Mark Miller) + +* SOLR-9037: Replace multiple "/replication" strings with one static constant. (Christine Poerschke) + +* SOLR-9047: zkcli should allow alternative locations for log4j configuration (Gregory Chanan) + +* SOLR-9066: Make CountMetric return long instead of double (Kevin Risden) + +* SOLR-9065, SOLR-9072, SOLR-9132: Migrate some distributed tests to SolrCloudTestCase. (Alan Woodward) + +* SOLR-8184: Negative tests for JDBC Connection String (Susheel Kumar, Jason Gerlowski, Kevin Risden) + +* SOLR-8458: Add Streaming Expressions tests for parameter substitution (Joel Bernstein, Cao Manh Dat, Dennis Gove, Kevin Risden) + +* SOLR-8467: CloudSolrStream and FacetStream should take a SolrParams object rather than a + Map<String, String> to allow more complex Solr queries to be specified. (Erick Erickson) + +* SOLR-9083: Remove all <types> and <fields> from schemas. NOTE: as in the JIRA I left a few in to insure the (no cost) + back compat. (Erick Erickson) + +* SOLR-9105: Fix a bunch of typos across 103 files (Bartosz Krasiński via janhoy) + +* SOLR-9159: New cloud based concurrent atomic update test (hossman) + +* SOLR-9119: several static methods in ValueSourceParser have been made private (hossman) + +* SOLR-9110: Move JoinFromCollection- SubQueryTransformer- BlockJoinFacet- Distrib Tests to SolrCloudTestCase (Mikhail Khludnev) + +* SOLR-9136: Separate out the error statistics into server-side error vs client-side error + (Jessica Cheng Mallet via Erick Erickson) + +* SOLR-9107: new @RandomizeSSL annotation for more fine grained control of SSL testing (hossman, sarowe) + +* SOLR-9081: Make SolrTestCaseJ4.beforeClass() / .afterClass() public so it + works with Mockito (Georg Sorst, Alan Woodward) + +* SOLR-8445: fix line separator in log4j.properties files (Ahmet Arslan via Mikhail Khludnev) + +* SOLR-2199: DataImportHandler (DIH) JdbcDataSource supports multiple resultsets per query (Kristine Jetzke, Mark Waddle via Mikhail Khludnev) + +================== 6.0.1 ================== + +Upgrade Notes +---------------------- + +* If you use historical dates, specifically on or before the year 1582, you should re-index. + +Bug Fixes +---------------------- + +* SOLR-8914: ZkStateReader's refreshLiveNodes(Watcher) is not thread safe. (Scott Blum, hoss, + sarowe, Erick Erickson, Mark Miller, shalin) + +* SOLR-9016: Fix SolrIdentifierValidator to not allow empty identifiers. (Shai Erera) + +* SOLR-8992: Restore Schema API GET method functionality removed in 6.0 (noble, Steve Rowe) + +* SOLR-9080, SOLR-9085: (6.0 bug) For years <= 1582, date math (round,add,sub) introduced error. Range faceting + on such dates was also affected. With this fixed, this is the first release range faceting works on BC years. + (David Smiley) + +* SOLR-8857: HdfsUpdateLog does not use configured or new default number of version buckets and is + hard coded to 256. (Mark Miller, yonik, Gregory Chanan) + +* SOLR-8902: Make sure ReturnFields only returns the requested fields from (fl=) evn when + DocumentTransformers ask for getExtraRequestFields() (ryan) + +* SOLR-8875: SolrCloud Overseer clusterState could unexpectedly be null resulting in NPE. + (Scott Blum via David Smiley) + +* SOLR-8946: bin/post failed to detect stdin usage on Ubuntu; maybe other unixes. (David Smiley) + +* SOLR-9004: Fix "name" field type definition in films example. (Alexandre Rafalovitch via Varun Thacker) + +* SOLR-8990: Fix top term links from schema browser page to use {!term} parser (hossman) + +* SOLR-8971: Preserve root cause when wrapping exceptions (hossman) + +* SOLR-9034: Atomic updates failed to work when there were copyField targets that had docValues + enabled. (Karthik Ramachandran, Ishan Chattopadhyaya, yonik) + +* SOLR-9028: Fixed some test related bugs preventing SSL + ClientAuth from ever being tested (hossman) + +* SOLR-9059: NPE in SolrClientCache following collection reload (Joel Bernstein, Ryan Yacyshyn) + +* SOLR-8792: ZooKeeper ACL support fixed. (Esther Quansah, Ishan Chattopadhyaya, Steve Rowe) + +* SOLR-9030: The 'downnode' overseer command can trip asserts in ZkStateWriter. + (Scott Blum, Mark Miller, shalin) + +* SOLR-9036: Solr slave is doing full replication (entire index) of index after master restart. + (Lior Sapir, Mark Miller, shalin) + +* SOLR-9058: Makes HashJoinStream and OuterHashJoinStream support different field names in the + incoming streams, eg. fieldA=fieldB. (Dennis Gove, Stephan Osthold) + +* SOLR-9093: Fix NullPointerException in TopGroupsShardResponseProcessor. (Christine Poerschke) + +* SOLR-9118: HashQParserPlugin should trim partition keys (Joel Bernstein) + +* SOLR-9117: The first SolrCore is leaked after reload. (Jessica Cheng Mallet via shalin) + +* SOLR-9116: Race condition causing occasional SolrIndexSearcher leak when SolrCore is reloaded. + (Jessica Cheng Mallet via shalin) + +* SOLR-8801: /bin/solr create script always returns exit code 0 when a collection/core already exists. + (Khalid Alharbi, Marius Grama via Steve Rowe) + +* SOLR-9134: Fix RestManager.addManagedResource return value. (Christine Poerschke) + +Other Changes +---------------------- +* SOLR-7516: Improve javadocs for JavaBinCodec, ObjectResolver and enforce the single-usage policy. + (Jason Gerlowski, Benoit Vanalderweireldt, shalin) + +* SOLR-8967: In SolrCloud mode, under the 'Core Selector' dropdown in the UI the Replication tab won't be displayed + anymore. The Replication tab is only beneficial to users running Solr in master-slave mode. (Varun Thacker) + +* SOLR-8985: Added back support for 'includeDynamic' flag to /schema/fields endpoint (noble) + +* SOLR-9131: Fix "start solr" text in cluster.vm Velocity template (janhoy) + +* SOLR-9053: Upgrade commons-fileupload to 1.3.1, fixing a potential vulnerability (Jeff Field, Mike Drob via janhoy) + +* SOLR-9115: Get rid of javax.xml.bind.DatatypeConverter in SimplePostTool + for Java 9 compatibility. (Uwe Schindler) + +* SOLR-5776,SOLR-9068,SOLR-8970: + - Refactor SSLConfig so that SSLTestConfig can provide SSLContexts using a NullSecureRandom + to prevent SSL tests from blocking on entropy starved machines. + - SSLTestConfig: Alternate (psuedo random) NullSecureRandom for Constants.SUN_OS. + - SSLTestConfig: Replace NullSecureRandom w/ NotSecurePsuedoRandom. + - Change SSLTestConfig to use a keystore file that is included as a resource in the + test-framework jar so users subclassing SolrTestCaseJ4 don't need to preserve magic paths. + (hossman) + +================== 6.0.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release + +Versions of Major Components +--------------------- +Apache Tika 1.7 +Carrot2 3.12.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.8.v20160314 + +System Requirements +---------------------- +* LUCENE-5950: Move to Java 8 as minimum Java version. + (Ryan Ernst, Uwe Schindler) + +Upgrading from Solr 5.x +---------------------- + +* The deprecated SolrServer and subclasses have been removed, use SolrClient + instead. + +* The deprecated <nrtMode> configuration in solrconfig.xml has been removed. + Please remove it from solrconfig.xml. + +* SolrClient.shutdown() has been removed, use SolrClient.close() instead. + +* The deprecated zkCredientialsProvider element in solrcloud section of solr.xml + is now removed. Use the correct spelling (zkCredentialsProvider) instead. + +* SOLR-7957: internal/expert - ResultContext was significantly changed and expanded + to allow for multiple full query results (DocLists) per Solr request. + TransformContext was rendered redundant and was removed. (yonik) + +* Several changes have been made regarding the "Similarity" used in Solr, in order to provide + better default behavior for new users. There are 3 key impacts of these changes on existing + users who upgrade: + * DefaultSimilarityFactory has been removed. If you currently have DefaultSimilarityFactory explicitly + referenced in your schema.xml, edit your config to use the functionally identical + ClassicSimilarityFactory. See SOLR-8239 for more details. + * The implicit default Similarity used when no <similarity/> is configured in schema.xml has + been changed to SchemaSimilarityFactory. Users who wish to preserve back-compatible behavior should + either explicitly configure ClassicSimilarityFactory, or ensure that the luceneMatchVersion + for the collection is less then 6.0. See SOLR-8270 + SOLR-8271 for details. + * SchemaSimilarityFactory has been modified to use BM25Similarity as the default for fieldTypes that + do not explicitly declare a Similarity. The legacy behavior of using ClassicSimilarity as the + default will occur if the luceneMatchVersion for the collection is less then 6.0, or the + 'defaultSimFromFieldType' configuration option may be used to specify any default of your choosing. + See SOLR-8261 + SOLR-8329 for more details. + +* If your solrconfig.xml file doesn't explicitly mention the schemaFactory to use then Solr will choose + the ManagedIndexSchemaFactory by default. Previously it would have chosen ClassicIndexSchemaFactory. + This means that the Schema APIs ( /<collection>/schema ) are enabled and the schema is mutable. + When Solr starts your schema.xml file will be renamed to managed-schema. If you want to retain the old behaviour + then please ensure that the solrconfig.xml explicitly uses the ClassicIndexSchemaFactory : + <schemaFactory class="ClassicIndexSchemaFactory"/> or your luceneMatchVersion in the solrconfig.xml is less than 6.0 + +* SolrIndexSearcher.QueryCommand and QueryResult were moved to their own classes. If you reference them + in your code, you should import them under o.a.s.search (or use your IDE's "Organize Imports"). + +* SOLR-8698: 'useParams' attribute specified in request handler cannot be overridden from request params + +* When requesting stats in date fields, "sum" is now a double value instead of a date. See SOLR-8671 + +* SOLR-8736: The deprecated GET methods for schema are now accessible through the bulk API. The output + has less details and is not backward compatible. + +* In the past, Solr guaranteed that retrieval of multi-valued fields would preserve the order of values. + Because values may now be retrieved from column-stored fields (docValues="true"), in conjunction with the + fact that docValues do not currently preserve order, means that users should set useDocValuesAsStored="false" + to prevent future optizations from using the column-stored values over the row-stored values when + fields have both stored="true" and docValues="true". + +* Formatted date-times from Solr have some differences. If the year is more than 4 digits, there is a leading '+'. + When there is a non-zero number of milliseconds, it is padded with zeros to 3 digits. Negative year (BC) dates are + now possible. Parsing: It is now an error to supply a portion of the date out of its, range, like 67 seconds. + +* SolrJ no longer includes DateUtil. If for some reason you need to format or parse dates, simply use Instant.format() + and Instant.parse(). + +* If you are using an RPT or other spatial field referencing Spatial4j in its configuration, then replace the string + "com.spatial4j.core" with "org.locationtech.spatial4j". Consider updating direct to Solr 6.2 which accepts the + old value, albeit with a warning. + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-3085: New edismax param mm.autoRelax which helps in certain cases of the stopwords/zero-hits issue (janhoy) + +* SOLR-7560: Parallel SQL Support (Joel Bernstein) + +* SOLR-7707: Add StreamExpression Support to RollupStream (Dennis Gove, Joel Bernstein) + +* SOLR-7903: Add the FacetStream to the Streaming API and wire it into the SQLHandler (Joel Bernstein) + +* SOLR-7986: JDBC Driver for SQL Interface (Uwe Schindler, Joel Bernstein) + +* SOLR-8038: Add the StatsStream to the Streaming API and wire it into the SQLHandler (Joel Bernstein) + +* SOLR-8086: Add support for SELECT DISTINCT queries to the SQL interface (Joel Bernstein) + +* SOLR-7543: Basic graph traversal query + Example: {!graph from="node_id" to="edge_id"}id:doc_1 + (Kevin Watters, yonik) + +* SOLR-6273: Cross Data Center Replication. Active/passive replication for separate + SolrClouds hosted on separate data centers. (Renaud Delbru, Yonik Seeley via Erick Erickson) + +* SOLR-7938: MergeStream now supports merging more than 2 streams together (Dennis Gove) + +* SOLR-8198: Change ReducerStream to use StreamEqualitor instead of StreamComparator (Dennis Gove) + +* SOLR-8268: StatsStream now implements the Expressible interface (Dennis Gove) + +* SOLR-7584: Adds Inner and LeftOuter Joins to the Streaming API and Streaming Expressions (Dennis Gove, Corey Wu) + +* SOLR-8188: Adds Hash and OuterHash Joins to the Streaming API and Streaming Expressions (Dennis Gove) + +* SOLR-7669: Add SelectStream and Tuple Operations to the Streaming API and Streaming Expressions (Dennis Gove) + +* SOLR-8337: Add ReduceOperation and wire it into the ReducerStream (Joel Bernstein) + +* SOLR-7904: Add StreamExpression Support to FacetStream (Dennis Gove) + +* SOLR-6398: Add IterativeMergeStrategy to support running Parallel Iterative Algorithms inside of Solr + (Joel Bernstein) + +* SOLR-8436: Real-time get now supports filters. (yonik) + +* SOLR-7535: Add UpdateStream to Streaming API and Streaming Expression (Jason Gerlowski, Joel Bernstein) + +* SOLR-8479: Add JDBCStream to Streaming API and Streaming Expressions for integration with external data sources + (Dennis Gove) + +* SOLR-8002: Add column alias support to the Parallel SQL Interface (Joel Bernstein) + +* SOLR-7525: Add ComplementStream and IntersectStream to the Streaming API and Streaming Expressions + (Dennis Gove, Jason Gerlowski, Joel Bernstein) + +* SOLR-8415: Provide command to switch between non/secure mode in ZK + (Mike Drob, Gregory Chanan) + +* SOLR-8556: Add ConcatOperation to be used with the SelectStream (Joel Bernstein, Dennis Gove) + +* SOLR-8550: Add asynchronous DaemonStreams to the Streaming API (Joel Bernstein) + +* SOLR-8285: Ensure the /export handler works with NULL field values (Joel Bernstein) + +* SOLR-8502: Improve Solr JDBC Driver to support SQL Clients like DBVisualizer (Kevin Risden, Joel Bernstein) + +* SOLR-8588: Add TopicStream to the streaming API to support publish/subscribe messaging (Joel Bernstein, Kevin Risden) + +* SOLR-8666: Adds header 'zkConnected' to response of SearchHandler and PingRequestHandler to notify the client when + a connection to zookeeper has been lost and there is a possibility of stale data on the node the request is coming + from. (Keith Laban, Dennis Gove) + +* SOLR-8522: Make it possible to use ip fragments in replica placement rules , such as ip_1, ip_2 etc (Arcadius Ahouansou, noble) + +* SOLR-8698: params.json can now specify 'appends' and 'invariants' (noble) + +* SOLR-8831: allow _version_ field to be retrievable via docValues (yonik) + + +Bug Fixes +---------------------- +* SOLR-8386: Add field option in the new admin UI schema page loads up even when no schemaFactory has been + explicitly specified since the default is ManagedIndexSchemaFactory. (Erick Erickson, Upayavira, Varun Thacker) + +* SOLR-8191: Guard against CloudSolrStream close method NullPointerException + (Kevin Risden, Joel Bernstein) + +* SOLR-8485: SelectStream now properly handles non-lowercase and/or quoted select field names (Dennis Gove) + +* SOLR-8525: Fix a few places that were failing to pass dimensional + values settings when copying a FieldInfo (Ishan Chattopadhyaya via + Mike McCandless) + +* SOLR-8409: Ensures that quotes in solr params (eg. q param) are properly handled (Dennis Gove) + +* SOLR-8640: CloudSolrClient does not send credentials for update request (noble, hoss) + +* SOLR-8461: CloudSolrStream and ParallelStream can choose replicas that are not active + (Cao Manh Dat, Varun Thacker, Joel Bernstein) + +* SOLR-8527: Improve JdbcTest to cleanup properly on failures (Kevin Risden, Joel Bernstein) + +* SOLR-8578: Successful or not, requests are not always fully consumed by Solrj clients and we + count on HttpClient or the JVM. (Mark Miller) + +* SOLR-8683: Always consume the full request on the server, not just in the case of an error. + (Mark Miller) + +* SOLR-8416: The collections create API should return after all replicas are active. + (Michael Sun, Mark Miller, Alexey Serba) + +* SOLR-8701: CloudSolrClient decides that there are no healthy nodes to handle a request too early. + (Mark Miller) + +* SOLR-8694: DistributedMap/Queue can create too many Watchers and some code simplification. + (Scott Blum via Mark Miller) + +* SOLR-8695: Ensure ZK watchers are not triggering our watch logic on connection events and + make this handling more consistent. (Scott Blum via Mark Miller) + +* SOLR-8633: DistributedUpdateProcess processCommit/deleteByQuery call finish on DUP and + SolrCmdDistributor, which violates the lifecycle and can cause bugs. (hossman via Mark Miller) + +* SOLR-8656: PeerSync should use same nUpdates everywhere. (Ramsey Haddad via Mark Miller) + +* SOLR-8697, SOLR-8837: Scope ZK election nodes by session to prevent elections from interfering with each other + and other small LeaderElector improvements. (Scott Blum via Mark Miller, Alan + Woodward) + +* SOLR-8599: After a failed connection during construction of SolrZkClient attempt to retry until a connection + can be made. (Keith Laban, Dennis Gove) + +* SOLR-8497: Merge index does not mark the Directory objects it creates as 'done' and they are retained in the + Directory cache. (Sivlio Sanchez, Mark Miller) + +* SOLR-8696: Start the Overseer before actions that need the overseer on init and when reconnecting after + zk expiration and improve init logic. (Scott Blum, Mark Miller) + +* SOLR-8420: Fix long overflow in sumOfSquares for Date statistics. (Tom Hill, Christine Poerschke, + Tomás Fernández Löbbe) + +* SOLR-8748: OverseerTaskProcessor limits number of concurrent tasks to just 10 even though the thread pool + size is 100. The limit has now been increased to 100. (Scott Blum, shalin) + +* SOLR-8375: ReplicaAssigner rejects valid nodes (Kelvin Tan, noble) + +* SOLR-8738: Fixed false success response when invalid deleteByQuery requests intially hit non-leader + cloud nodes (hossman) + +* SOLR-8771: Multi-threaded core shutdown creates executor per core. (Mike Drob via Mark Miller) + +* SOLR-8145: Fix position of OOM killer script when starting Solr in the background (Jurian Broertjes via + Timothy Potter) + +* SOLR-8769: Fix document exclusion in mlt query parser in Cloud mode for schemas that have non-"id" + unique field (Erik Hatcher, Anshum Gupta) + +* SOLR-8728: ReplicaAssigner throws NPE when a partial list of nodes are only participating in replica + placement. splitshard should preassign nodes using rules, if rules are present (noble, Shai Erera) + +* SOLR-8779: Fix missing InterruptedException handling in ZkStateReader.java (Varun Thacker) + +* SOLR-8449: Fix the core restore functionality to allow restoring multiple times on the same core + (Johannes Brucher, Varun Thacker) + +* SOLR-8155: JSON Facet API - field faceting on a multi-valued string field without + docValues (i.e. UnInvertedField implementation), but with a prefix or with a sort + other than count, resulted in incorrect results. This has been fixed, and facet.prefix + support for facet.method=uif has been enabled. (Mikhail Khludnev, yonik) + +* SOLR-8790: Collections API responses contain node name in the core-level responses that are + returned. (Anshum Gupta) + +* SOLR-8804: Fix a race condition in the ClusterStatus API call whereby the call would fail when a concurrent delete + collection api command was executed (Alexey Serba, Varun Thacker) + +* SOLR-8835: JSON Facet API: fix faceting exception on multi-valued numeric fields that + have docValues. (yonik) + +* SOLR-8838: Returning non-stored docValues is incorrect for negative floats and doubles. + (Ishan Chattopadhyaya, Steve Rowe) + +* SOLR-8867: {!frange} queries will now avoid matching documents without a value in the + numeric field. For more complex functions, FunctionValues.exists() must also return true + for the document to match. (yonik) + +* SOLR-8886: Fix TrieField.toObject(IndexableField) to work for field with docValues + enabled. (yonik) + +* SOLR-8891: Fix StrField.toObject and toExternal to work with docValue IndexableField + instances. (yonik) + +* SOLR-8865: Real-time get sometimes fails to retrieve stored fields from docValues. + (Ishan Chattopadhyaya, yonik) + +Optimizations +---------------------- +* SOLR-7876: Speed up queries and operations that use many terms when timeAllowed has not been + specified. Speedups of up to 8% were observed. (yonik) + +* SOLR-8037: Speed up creation of filters from term range queries (i.e. non-numeric range queries) + and use the filter cache for term range queries that are part of larger queries. Some observed + speedups were up to 2.5x for production of filters, and up to 10x for query evaluation with + embedded term range queres that resulted in filter cache hits. (yonik) + +* SOLR-8559: FCS facet performance optimization which significantly speeds up processing when terms + are high cardinality and the matching docset is small. When facet minCount > 0 and the number of + matching documents is small (or 0) this enhancement prevents considering terms which have a 0 + count. Also includes change to move to the next non-zero term value when selecting a segment + position. (Keith Laban, Steve Bower, Dennis Gove) + +* SOLR-8532: Optimize GraphQuery when maxDepth is set by not collecting edges at the maxDepth level. + (Kevin Watters via yonik) + +* SOLR-8669: Non binary responses use chunked encoding because we flush the outputstream early. + (Mark Miller) + +* SOLR-8720: ZkController#publishAndWaitForDownStates should use #publishNodeAsDown. (Mark Miller) + +* SOLR-8082: Can't query against negative float or double values when indexed="false" + docValues="true" multiValued="false". (hossman, Ishan Chattopadhyaya, yonik, Steve Rowe) + +Other Changes +---------------------- + +* SOLR-6127: Improve example docs, using films data (Varun Thacker via ehatcher) + +* SOLR-6895: Deprecated SolrServer classes have been removed (Alan Woodward, + Erik Hatcher) + +* SOLR-6954: Deprecated SolrClient.shutdown() method removed (Alan Woodward) + +* SOLR-7355: Switch from Google's ConcurrentLinkedHashMap to Caffeine. Only + affects HDFS support. (Ben Manes via Shawn Heisey) + +* SOLR-7624: Remove deprecated zkCredientialsProvider element in solrcloud section of solr.xml. + (Xu Zhang, Per Steffensen, Ramkumar Aiyengar, Mark Miller) + +* SOLR-7513: Add Equalitors to Streaming Expressions (Dennis Gove, Joel Bernstein) + +* SOLR-7528: Simplify Interfaces used in Streaming Expressions (Dennis Gove, Joel Bernstein) + +* SOLR-7554: Add checks in Streams for incoming stream order (Dennis Gove, Joel Bernstein) + +* SOLR-7441: Improve overall robustness of the Streaming stack: Streaming API, + Streaming Expressions, Parallel SQL (Joel Bernstein) + +* SOLR-8153: Support upper case and mixed case column identifiers in the SQL interface + (Joel Bernstein) + +* SOLR-8132: HDFSDirectoryFactory now defaults to using the global block cache. (Mark Miller) + +* SOLR-8261: Change SchemaSimilarityFactory default to BM25Similarity (hossman) + +* SOLR-8259: Remove deprecated JettySolrRunner.getDispatchFilter() + +* SOLR-8258: Change default hdfs tlog replication factor from 1 to 3. (Mark Miller) + +* SOLR-8270: Change implicit default Similarity to use BM25 when luceneMatchVersion >= 6 (hossman) + +* SOLR-8271: Change implicit default Similarity to use SchemaSimilarityFactory when luceneMatchVersion >= 6 (hossman) + +* SOLR-8179: SQL JDBC - DriverImpl loadParams doesn't support keys with no values in the connection string + (Kevin Risden, Joel Bernstein) + +* SOLR-8131: Make ManagedIndexSchemaFactory the default schemaFactory when luceneMatchVersion >= 6 + (Uwe Schindler, shalin, Varun Thacker) + +* SOLR-8266: Remove Java Serialization from the Streaming API. The /stream handler now only accepts + Streaming Expressions. (Jason Gerlowski, Joel Bernstein) + +* SOLR-8426: Enable /export, /stream and /sql handlers by default and remove them from example configs. (shalin) + +* SOLR-8443: Change /stream handler http param from "stream" to "expr" (Joel Bernstein, Dennis Gove) + +* SOLR-5209: Unloading or deleting the last replica of a shard now no longer + cascades to remove the shard from the clusterstate. (Christine Poerschke) + +* SOLR-8190: Implement Closeable on TupleStream (Kevin Risden, Joel Bernstein) + +* SOLR-8529: Improve JdbcTest to not use plain assert statements (Kevin Risden, Joel Bernstein) + +* SOLR-7339: Upgrade Jetty to v9.3.8.v20160314. (Gregg Donovan, shalin, Mark Miller, Steve Rowe) + +* SOLR-5730: Make Lucene's SortingMergePolicy and EarlyTerminatingSortingCollector configurable in Solr. + (Christine Poerschke, hossmann, Tomás Fernández Löbbe, Shai Erera) + +* SOLR-8677: Prevent shards containing invalid characters from being created. Checks added server-side + and in SolrJ. (Shai Erera, Jason Gerlowski, Anshum Gupta) + +* SOLR-8693: Improve ZkStateReader logging. (Scott Blum via Mark Miller) + +* SOLR-8710: Upgrade morfologik-stemming to version 2.1.0. (Dawid Weiss) + +* SOLR-8711: Upgrade Carrot2 clustering dependency to 3.12.0. (Dawid Weiss) + +* SOLR-8690: Make peersync fingerprinting optional with solr.disableFingerprint system + property. (yonik) + +* SOLR-8691: Cache index fingerprints per searcher. (yonik) + +* SOLR-8746: Renamed Overseer.getInQueue to getStateUpdateQueue, getInternalQueue to getInternalWorkQueue + and added javadocs. (Scott Blum, shalin) + +* SOLR-8752: Add a test for SizeLimitedDistributedMap and improve javadocs. (shalin) + +* SOLR-8671: Date statistics: make "sum" a double instead of a long/date (Tom Hill, Christine Poerschke, + Tomás Fernández Löbbe) + +* SOLR-8713: new UI and example solrconfig files point to Reference Guide for Solr Query Syntax instead + of the wiki. (Marius Grama via Tomás Fernández Löbbe) + +* SOLR-8758: Add a new SolrCloudTestCase class, using MiniSolrCloudCluster (Alan + Woodward) + +* SOLR-8736: schema GET operations on fields, dynamicFields, fieldTypes, copyField are + reimplemented as a part of the bulk API with less details (noble) + +* SOLR-8766: Remove deprecated <admin> tag in solrconfig.xml and support for admin/gettableFiles + (noble, Jason Gerlowski, Varun Thacker) + +* SOLR-8799: Improve error message when tuple can't be read by SolrJ JDBC (Kevin Risden, Joel Bernstein) + +* SOLR-8836: Return 400, and a SolrException when an invalid json is provided to the update handler + instead of 500. (Jason Gerlowski via Anshum Gupta) + +* SOLR-8740: docValues are now enabled by default for most non-text (string, date, and numeric) fields + in the schema templates. (yonik) + +* SOLR-8819: Implement DatabaseMetaDataImpl getTables() and fix getSchemas(). (Trey Cahill, + Joel Bernstein, Kevin Risden) + +* SOLR-8810: Implement Connection.setReadOnly, Statement.set/getFetchSize, + ResultSet.getType (Kevin Risden) + +* SOLR-8904: All dates are formatted via Instant.toString() (ISO-8601); see Solr upgrade notes for differences. Will + now parse (and format) dates with a leading '+' or '-' (BC dates or dates > 4 digit year. + [value] and ms() and contrib/analytics now parse with date math. (David Smiley) + +* SOLR-8904: DateUtil in SolrJ moved to the extraction contrib as ExtractionDateUtil. Obsolete methods were removed. + (David Smiley) + +======================= 5.5.5 ======================= + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.7 +Carrot2 3.10.4 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.2.13.v20150730 + +Upgrade Notes +---------------------- + +* SOLR-11477: in the XML query parser (defType=xmlparser or {!xmlparser ... }) + the resolving of external entities is now disallowed by default. + +* SOLR-11482: RunExecutableListener was deprecated and is disabled by default for + security reasons. Legacy applications still using it must explicitely pass + '-Dsolr.enableRunExecutableListener=true' to the Solr command line. + Be aware that you should really disable API-based config editing at the same + time, using '-Ddisable.configEdit=true'! (Uwe Schindler) + +Bug Fixes +---------------------- + +* SOLR-10420: Leaking one SolrZkClient instance per second (Scott Blum, Cao Manh Dat, Markus Jelsma, Steve Rowe) + +* SOLR-11477: Disallow resolving of external entities in the XML query parser (defType=xmlparser). + (Michael Stepankin, Olga Barinova, Uwe Schindler, Christine Poerschke) + +Other +---------------------- + +* SOLR-11486: Update jmatio to 1.2 and prevent deserialization inside Matlab files. + (Uwe Schindler, Steve Rowe) + +======================= 5.5.4 ======================= + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.15.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.14.v20161028 + + +Other Changes +---------------------- + +* SOLR-9819: Upgrade commons-fileupload to 1.3.2, fixing a potential vulnerability CVE-2016-3092 (Anshum Gupta) + +* SOLR-10031: Validation of filename params in ReplicationHandler (Hrishikesh Gadre, janhoy) + + +======================= 5.5.3 ======================= + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.13 +Carrot2 3.12.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.3.8.v20160314 + + +(No Changes) + + +======================= 5.5.2 ======================= + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +Versions of Major Components +--------------------- +Apache Tika 1.7 +Carrot2 3.10.4 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.2.13.v20150730 + +Bug Fixes +--------------------- + +* SOLR-8695: Ensure ZK watchers are not triggering our watch logic on connection events and + make this handling more consistent. (Scott Blum via Mark Miller) + +* SOLR-9198: config APIs unable to add multiple values with same name (noble) + +* SOLR-9191: OverseerTaskQueue.peekTopN() fatally flawed (Scott Blum, Noble Paul) + +* SOLR-8812: edismax: turn off mm processing if no explicit mm spec is provided + and there are explicit operators (except for AND) - addresses problems caused by SOLR-2649. + (Greg Pendlebury, Jan Høydahl, Erick Erickson, Steve Rowe) + +* SOLR-9034: Atomic updates failed to work when there were copyField targets that had docValues + enabled. (Karthik Ramachandran, Ishan Chattopadhyaya, yonik) + +* SOLR-8940: Fix group.sort option (hossman) + +* SOLR-8857: HdfsUpdateLog does not use configured or new default number of version buckets and is + hard coded to 256. (Mark Miller, yonik, Gregory Chanan) + +* SOLR-8875: SolrCloud Overseer clusterState could unexpectedly be null resulting in NPE. + (Scott Blum via David Smiley) + +* SOLR-8946: bin/post failed to detect stdin usage on Ubuntu; maybe other unixes. (David Smiley) + +* SOLR-9004: Fix "name" field type definition in films example. (Alexandre Rafalovitch via Varun Thacker) + +* SOLR-8990: Fix top term links from schema browser page to use {!term} parser (hossman) + +* SOLR-8971: Preserve root cause when wrapping exceptions (hossman) + +* SOLR-8792: ZooKeeper ACL support fixed. (Esther Quansah, Ishan Chattopadhyaya, Steve Rowe) + +* SOLR-9030: The 'downnode' overseer command can trip asserts in ZkStateWriter. + (Scott Blum, Mark Miller, shalin) + +* SOLR-9036: Solr slave is doing full replication (entire index) of index after master restart. + (Lior Sapir, Mark Miller, shalin) + +* SOLR-9093: Fix NullPointerException in TopGroupsShardResponseProcessor. (Christine Poerschke) + +* SOLR-9118: HashQParserPlugin should trim partition keys (Joel Bernstein) + +* SOLR-9117: The first SolrCore is leaked after reload. (Jessica Cheng Mallet via shalin) + +* SOLR-9116: Race condition causing occasional SolrIndexSearcher leak when SolrCore is reloaded. + (Jessica Cheng Mallet via shalin) + +* SOLR-8801: /bin/solr create script always returns exit code 0 when a collection/core already exists. + (Khalid Alharbi, Marius Grama via Steve Rowe) + +* SOLR-9134: Fix RestManager.addManagedResource return value. (Christine Poerschke) + +* SOLR-9151: Fix SolrCLI so that bin/solr -e cloud example can be run from any CWD (janhoy) + +* SOLR-9165: Spellcheck does not return collations if "maxCollationTries" is used with "cursorMark". + (James Dyer) + +* SOLR-8612: closing JDBC Statement on failures in DataImportHandler (DIH) (Kristine Jetzke via Mikhail Khludnev) + +* SOLR-8676: keep LOG4J_CONFIG in solr.cmd (Kristine Jetzke via Mikhail Khludnev) + +* SOLR-9176: facet method ENUM was sometimes unnecessarily being rewritten to + FCS, causing slowdowns (Alessandro Benedetti, Jesse McLaughlin, Alan Woodward) + +Other Changes +---------------------- + +* SOLR-7516: Improve javadocs for JavaBinCodec, ObjectResolver and enforce the single-usage policy. + (Jason Gerlowski, Benoit Vanalderweireldt, shalin) + +* SOLR-8967: In SolrCloud mode, under the 'Core Selector' dropdown in the UI the Replication tab won't be displayed + anymore. The Replication tab is only beneficial to users running Solr in master-slave mode. (Varun Thacker) + +* SOLR-9131: Fix "start solr" text in cluster.vm Velocity template (janhoy) + +* SOLR-9053: Upgrade commons-fileupload to 1.3.1, fixing a potential vulnerability (Jeff Field, Mike Drob via janhoy) + +* SOLR-8866: UpdateLog will now throw an exception if it doesn't know how to serialize a value. + (David Smiley) + +* SOLR-8933: Solr should not close container streams. (Mike Drob, Uwe Schindler, Mark Miller) + +* SOLR-9037: Replace multiple "/replication" strings with one static constant. (Christine Poerschke) + +* SOLR-9047: zkcli should allow alternative locations for log4j configuration (Gregory Chanan) + +* SOLR-9105: Fix a bunch of typos across 103 files (Bartosz Krasiński via janhoy) + +* SOLR-8445: fix line separator in log4j.properties files (Ahmet Arslan via Mikhail Khludnev) + +* SOLR-8674: Stop ignoring sysprop solr.tests.mergePolicy, and make tests randomly choose between + setting <mergePolicy> and <mergePolicyFactory>, which was added in SOLR-8621. (Christine Poerschke) + +======================= 5.5.1 ======================= + +Bug Fixes +---------------------- + +* SOLR-8737: Managed synonym lists do not include the original term in the expand (janhoy) + +* SOLR-8734: fix (maxMergeDocs|mergeFactor) deprecation warnings: in solrconfig.xml + <maxMergeDocs|mergeFactor> may not be combined with <mergePolicyFactory> and + <maxMergeDocs|mergeFactor> on their own or combined with <mergePolicy> is a warning. + (Christine Poerschke, Shai Erera) + +* SOLR-8712: Variable solr.core.instanceDir was not being resolved (Kristine + Jetzke, Shawn Heisey, Alan Woodward) + +* SOLR-8793: Fix Core admin status API to not fail when computing the size of the segments_N + file if the file no longer exists (for example, if a commit happened and the IndexReader + hasn't refreshed yet). In this case the reported size of the file is -1. + (Shai Erera, Alexey Serba, Richard Coggins) + +* SOLR-8728: ReplicaAssigner throws NPE when a partial list of nodes are only participating in replica + placement. splitshard should preassign nodes using rules, if rules are present (noble, Shai Erera) + +* SOLR-8838: Returning non-stored docValues is incorrect for negative floats and doubles. + (Ishan Chattopadhyaya, Steve Rowe) + +* SOLR-8870: AngularJS Query tab no longer URL-encodes the /select part of the request, fixing possible 404 issue + when Solr is behind a proxy. Also, now supports old-style &qt param when handler not prefixed with "/" (janhoy) + +* SOLR-8725: Allow hyphen in collection, core, shard, and alias name as the non-first character + (Anshum Gupta) (from 6.0) + +* SOLR-8155: JSON Facet API - field faceting on a multi-valued string field without + docValues (i.e. UnInvertedField implementation), but with a prefix or with a sort + other than count, resulted in incorrect results. This has been fixed, and facet.prefix + support for facet.method=uif has been enabled. (Mikhail Khludnev, yonik) + +* SOLR-8835: JSON Facet API: fix faceting exception on multi-valued numeric fields that + have docValues. (yonik) + +* SOLR-8908: Fix to OnReconnect listener registration to allow listeners to deregister, such + as when a core is reloaded or deleted to avoid a memory leak. (Timothy Potter) + +* SOLR-8662: SchemaManager waits correctly for replicas to be notified of a new change + (sarowe, Noble Paul, Varun Thacker) + +* SOLR-9007: Remove mention of the managed_schema_configs as valid config directory when creating + the collection for the SolrCloud example. (Timothy Potter) + +* SOLR-8983: Cleanup clusterstate and replicas for a failed create collection request + (Varun Thacker, Anshum Gupta) + +* SOLR-8578: Successful or not, requests are not always fully consumed by Solrj clients and we + count on HttpClient or the JVM. (Mark Miller) + +* SOLR-8683: Always consume the full request on the server, not just in the case of an error. + (Mark Miller) + +* SOLR-8701: CloudSolrClient decides that there are no healthy nodes to handle a request too early. + (Mark Miller) + +* SOLR-8416: The collections create API should return after all replicas are active. + (Michael Sun, Mark Miller, Alexey Serba) + +* SOLR-8914: ZkStateReader's refreshLiveNodes(Watcher) is not thread safe. (Scott Blum, hoss, + sarowe, Erick Erickson, Mark Miller, shalin) + +* SOLR-8973: Zookeeper frenzy when a core is first created. (Janmejay Singh, Scott Blum, shalin) + +* SOLR-8694: DistributedMap/Queue can create too many Watchers and some code simplification. + (Scott Blum via Mark Miller) + +* SOLR-8633: DistributedUpdateProcess processCommit/deleteByQuery call finish on DUP and + SolrCmdDistributor, which violates the lifecycle and can cause bugs. (hossman via Mark Miller) + +* SOLR-8656: PeerSync should use same nUpdates everywhere. (Ramsey Haddad via Mark Miller) + +* SOLR-8697, SOLR-8837: Scope ZK election nodes by session to prevent elections from interfering with each other + and other small LeaderElector improvements. (Scott Blum via Mark Miller, Alan + Woodward) + +* SOLR-8599: After a failed connection during construction of SolrZkClient attempt to retry until a connection + can be made. (Keith Laban, Dennis Gove) + +* SOLR-8420: Fix long overflow in sumOfSquares for Date statistics. (Tom Hill, Christine Poerschke, + Tomás Fernández Löbbe) + +* SOLR-8748: OverseerTaskProcessor limits number of concurrent tasks to just 10 even though the thread pool + size is 100. The limit has now been increased to 100. (Scott Blum, shalin) + +* SOLR-8375: ReplicaAssigner rejects valid nodes (Kelvin Tan, noble) + +* SOLR-8738: Fixed false success response when invalid deleteByQuery requests intially hit non-leader + cloud nodes (hossman) + +* SOLR-8771: Multi-threaded core shutdown creates executor per core. (Mike Drob via Mark Miller) + +* SOLR-8145: Fix position of OOM killer script when starting Solr in the background (Jurian Broertjes via + Timothy Potter) + +* SOLR-8769: Fix document exclusion in mlt query parser in Cloud mode for schemas that have non-"id" + unique field (Erik Hatcher, Anshum Gupta) + +* SOLR-8728: ReplicaAssigner throws NPE when a partial list of nodes are only participating in replica + placement. splitshard should preassign nodes using rules, if rules are present (noble, Shai Erera) + +* SOLR-8779: Fix missing InterruptedException handling in ZkStateReader.java (Varun Thacker) + +* SOLR-8449: Fix the core restore functionality to allow restoring multiple times on the same core + (Johannes Brucher, Varun Thacker) + +* SOLR-8790: Collections API responses contain node name in the core-level responses that are + returned. (Anshum Gupta) + +* SOLR-8804: Fix a race condition in the ClusterStatus API call whereby the call would fail when a concurrent delete + collection api command was executed (Alexey Serba, Varun Thacker) + +* SOLR-9016: Fix SolrIdentifierValidator to not allow empty identifiers. (Shai Erera) + +* SOLR-8886: Fix TrieField.toObject(IndexableField) to work for field with docValues + enabled. (yonik) + +* SOLR-8891: Fix StrField.toObject and toExternal to work with docValue IndexableField + instances. (yonik) + +* SOLR-8865: Real-time get sometimes fails to retrieve stored fields from docValues. + (Ishan Chattopadhyaya, yonik) + +* SOLR-9046: Fix solr.cmd that wrongly assumes Jetty will always listen on 0.0.0.0. + (Bram Van Dam, Uwe Schindler) + +Other Changes +---------------------- + +* SOLR-8758: Add a new SolrCloudTestCase class, using MiniSolrCloudCluster (Alan + Woodward) + +======================= 5.5.0 ======================= + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release + +Versions of Major Components +--------------------- +Apache Tika 1.7 +Carrot2 3.10.4 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.2.13.v20150730 + +Upgrading from Solr 5.4 +----------------------- + +* The Solr schema version has been increased to 1.6. Since schema version 1.6, all non-stored docValues fields + will be returned along with other stored fields when all fields (or pattern matching globs) are specified + to be returned (e.g. fl=*) for search queries. This behavior can be turned on and off by setting + 'useDocValuesAsStored' parameter for a field or a field type to true (default since schema version 1.6) + or false (default till schema version 1.5). + Note that enabling this property has performance implications because DocValues are column-oriented and may + therefore incur additional cost to retrieve for each returned document. All example schema are upgraded to + version 1.6 but any older schemas will default to useDocValuesAsStored=false and continue to work as in + older versions of Solr. If this new behavior is desirable, then you should set version attribute in your + schema file to '1.6'. Re-indexing is not necessary to upgrade the schema version. + Also note that while returning non-stored fields from docValues (default in schema versions 1.6+, unless + useDocValuesAsStored is false), the values of a multi-valued field are returned in sorted order. + If you require the multi-valued fields to be returned in the original insertion order, then make your + multi-valued field as stored. This requires re-indexing. + See SOLR-8220 for more details. + +* All protected methods from CoreAdminHandler other than handleCustomAction() is removed by SOLR-8476 and can + no more be overridden. If you still wish to override those methods, override the handleRequestBody() + +* The PERSIST CoreAdmin action which was a NOOP and returned a deprecated message has been removed. See SOLR-8476 + for more details. The corresponding SolrJ action has also been removed. + +* bin/post now defaults application/json files to the /update/json/docs end-point. Use `-format solr` to force + files to the /update end-point. See SOLR-7042 for more details. + +* In solrconfig.xml the <mergePolicy> element is deprecated in favor of a similar <mergePolicyFactory> element, + the <mergeFactor> and <maxMergeDocs> elements are also deprecated, please see SOLR-8621 for full details. + + To migrate your existing solrconfig.xml, you can replace elements as follows: + + <!-- TieredMergePolicy example --> + <!-- deprecated --> + <mergeFactor>??</mergeFactor> + <mergePolicy class="org.apache.lucene.index.TieredMergePolicy"> + <bool name="useCompoundFile">???</bool> <!-- deprecated since Lucene/Solr 4.4.0 --> + ... + </mergePolicy> + <!-- replacement --> + <useCompoundFile>???</useCompoundFile> <!-- since Lucene/Solr 4.4.0 --> + <mergePolicyFactory class="org.apache.solr.index.TieredMergePolicyFactory"> + <int name="maxMergeAtOnce">??</int> <!-- formerly the <mergeFactor> element implicitly set this --> + <int name="segmentsPerTier">??</int> <!-- formerly the <mergeFactor> element implicitly set this --> + ... + </mergePolicyFactory> + + <!-- Log(ByteSize|Doc)MergePolicy example --> + <!-- deprecated --> + <maxMergeDocs>????</maxMergeDocs> + <mergeFactor>??</mergeFactor> + <mergePolicy class="org.apache.lucene.index.Log?MergePolicy"> + <bool name="useCompoundFile">???</bool> <!-- deprecated since Lucene/Solr 4.4.0 --> + ... + </mergePolicy> + <!-- replacement --> + <useCompoundFile>???</useCompoundFile> <!-- since Lucene/Solr 4.4.0 --> + <mergePolicyFactory class="org.apache.solr.index.Log?MergePolicyFactory"> + <int name="maxMergeDocs">????</int> <!-- formerly the <maxMergeDocs> element set this --> + <int name="mergeFactor">??</int> <!-- formerly the <mergeFactor> element set this --> + ... + </mergePolicyFactory> + +* Clearing up stored async collection api responses via REQUESTSTATUS call is now deprecated and would be + removed in 6.0. See SOLR-8648 for more details. + +* SOLR-6594: Deprecated the old schema API which will be removed in a later major release + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-7928: Improve CheckIndex to work against HdfsDirectory + (Mike Drob, Gregory Chanan) + +* SOLR-8378: Add upconfig and downconfig commands to the bin/solr script + (Erick Erickson) + +* SOLR-8434: Add wildcard support to role, to match any role in RuleBasedAuthorizationPlugin (noble) + +* SOLR-4280: Allow specifying "spellcheck.maxResultsForSuggest" as a percentage of filter + query results (Markus Jelsma via James Dyer) + +* SOLR-8429: Add a flag 'blockUnknown' to BasicAuthPlugin to block unauthenticated requests (noble) + +* SOLR-8230: JSON Facet API: add "facet-info" into debug section of response when debugQuery=true + (Michael Sun, yonik) + +* SOLR-8428: RuleBasedAuthorizationPlugin adds an 'all' permission (noble) + +* SOLR-5743: BlockJoinFacetComponent and BlockJoinDocSetFacetComponent for calculating facets by + child.facet.field parameter with {!parent ..}.. query. They count facets on children documents + aggregating (deduplicating) counts by parent documents (Dr. Oleg Savrasov via Mikhail Khludnev) + +* SOLR-8220: Read field from DocValues for non stored fields. + (Keith Laban, yonik, Erick Erickson, Ishan Chattopadhyaya, shalin) + +* SOLR-8470: Make TTL of PKIAuthenticationPlugin's tokens configurable through a system property + (pkiauth.ttl) (noble) + +* SOLR-8477: Let users choose compression mode in SchemaCodecFactory (Tomás Fernández Löbbe) + +* SOLR-839: XML QueryParser support (defType=xmlparser) + Lucene includes a queryparser that supports the creation of Lucene queries from XML. + The queries supported by lucene.queryparser.xml.CoreParser are now supported by the newly + created solr.search.SolrCoreParser and in future SolrCoreParser could support additional + queries also. + Example: <BooleanQuery fieldName="description"> + <Clause occurs="must"> <TermQuery>shirt</TermQuery> </Clause> + <Clause occurs="mustnot"> <TermQuery>plain</TermQuery> </Clause> + <Clause occurs="should"> <TermQuery>cotton</TermQuery> </Clause> + <Clause occurs="must"> + <BooleanQuery fieldName="size"> + <Clause occurs="should"> <TermsQuery>S M L</TermsQuery> </Clause> + </BooleanQuery> + </Clause> + </BooleanQuery> + (Erik Hatcher, Karl Wettin, Daniel Collins, Nathan Visagan, Ahmet Arslan, Christine Poerschke) + +* SOLR-8312: Add domain size and numBuckets to facet telemetry info (facet debug info + for the new Facet Module). (Michael Sun, yonik) + +* SOLR-8534: Add generic support for collection APIs to be async. Thus more actions benefit from having async + support. The commands that additionally get async support are: delete/reload collection, create/delete alias, + create/delete shard, delete replica, add/delete replica property, add/remove role, + overseer status, balance shard unique, rebalance leaders, modify collection, migrate state format (Varun Thacker) + +* SOLR-4619: Improve PreAnalyzedField query analysis. (Andrzej Bialecki, Steve Rowe) + +* SOLR-8560: Added RequestStatusState enum which can be used when comparing states of + asynchronous requests. (Shai Erera) + +* SOLR-8586: added index fingerprint, a hash over all versions currently in the index. + PeerSync now uses this to check if replicas are in sync. (yonik) + +* SOLR-8500: Allow the number of threads ConcurrentUpdateSolrClient StreamingSolrClients configurable by a + system property. NOTE: this is an expert option and can result in more often needing to do full index replication + for recovery, the sweet spot for using this is very high volume, leader-only indexing. (Tim Potter, Erick Erickson) + +* SOLR-8642: SOLR allows creation of collections with invalid names + (Jason Gerlowski via Erick Erickson) + +* SOLR-8621: Deprecate <mergePolicy> in favor of <mergePolicyFactory>. It allows to configure + both the "simple" merge policies, but also more advanced ones, e.g. UpgradeIndexMergePolicy. + (Christine Poerschke, Shai Erera) + +* SOLR-8648: DELETESTATUS API for selective deletion and flushing of stored async collection API responses. + (Anshum Gupta) + +* SOLR-8466: adding facet.method=uif to bring back UnInvertedField faceting which is used to work on + facet.method=fc. It's more performant for rarely changing indexes. Note: it ignores prefix and contains yet. + (Jamie Johnson via Mikhail Khludnev) + +Bug Fixes +---------------------- + +* SOLR-8175: Word Break Spellchecker would throw AIOOBE with certain queries containing + "should" clauses. (Ryan Josal via James Dyer) + +* SOLR-2556: The default spellcheck query converter was ignoring terms consisting entirely + of digits. (James Dyer) + +* SOLR-8366: ConcurrentUpdateSolrClient attempts to use response's content type as charset encoding + for parsing exception. (shalin) + +* SOLR-6271: Fix ConjunctionSolrSpellChecker to not compare StringDistance by instance. + (Igor Kostromin via James Dyer) + +* SOLR-7304: Fix Spellcheck Collate to not invalidate range queries. (James Dyer) + +* SOLR-8373: KerberosPlugin: Using multiple nodes on same machine leads clients to + fetch TGT for every request (Ishan Chattopadhyaya via noble) + +* SOLR-8367: Fix the LeaderInitiatedRecovery 'all replicas participate' fail-safe. + (Mark Miller, Mike Drob) + +* SOLR-8401: Windows start script fails when executed from a different drive. + (Nicolas Gavalda via Erick Erickson) + +* SOLR-6992: Fix "Files" UI to show the managed-schema file as well. + (Shawn Heisey, Varun Thacker) + +* SOLR-2649: MM ignored in edismax queries with operators. + (Greg Pendlebury, Jan Høydahl et. al. via Erick Erickson) + +* SOLR-8372: Canceled recovery can rarely lead to inconsistent shards: + If a replica is recovering via index replication, and that recovery fails + (for example if the leader goes down), and then some more updates are received + (there could be a few left to be processed from the leader that just went down), + and then that replica is brought down, it will think it is up-to-date when + restarted. (shalin, Mark Miller, yonik) + +* SOLR-8419: TermVectorComponent for distributed search when distrib.singlePass could include term + vectors for documents that matched the query yet weren't in the returned documents. (David Smiley) + +* SOLR-8015: HdfsLock may fail to close a FileSystem instance if it cannot immediately + obtain an index lock. (Mark Miller) + +* SOLR-8422: When authentication enabled, requests fail if sent to a node that doesn't host + the collection (noble) + +* SOLR-8059: &debug=results for distributed search when distrib.singlePass (sometimes activated + automatically) could result in an NPE. (David Smiley, Markus Jelsma) + +* SOLR-8460: /analysis/field could throw exceptions for custom attributes. (David Smiley, Uwe Schindler) + +* SOLR-8276: Atomic updates and realtime-get do not work with non-stored docvalues. + (Ishan Chattopadhyaya, yonik via shalin) + +* SOLR-7462: AIOOBE in RecordingJSONParser (Scott Dawson, noble) + +* SOLR-8494: SimplePostTool and therefore the bin/post script cannot upload files larger than 2.1GB. (shalin) + +* SOLR-8451: We should not call method.abort in HttpSolrClient or HttpSolrCall#remoteQuery and + HttpSolrCall#remoteQuery should not close streams. (Mark Miller) + +* SOLR-8450: Our HttpClient retry policy is too permissive. (Mark Miller, shalin) + +* SOLR-8533: Raise default maxUpdateConnections and maxUpdateConnectionsPerHost to 100k each. + (Mark Miller) + +* SOLR-8453: Solr should attempt to consume the request inputstream on errors as we cannot + count on the container to do it. (Mark Miller, Greg Wilkins, yonik, Joakim Erdfelt) + +* SOLR-6279: cores?action=UNLOAD now waits for the core to close before unregistering it from ZK. + (Christine Poerschke) + +* SOLR-2798: Fixed local params to work correctly with multivalued params + (Demian Katz via hossman) + +* SOLR-8541: Highlighting a geo RPT field would throw an NPE instead of doing nothing. + (Pawel Rog via David Smiley) + +* SOLR-8548: Core discovery was not following symlinks (Aaron LaBella via Alan + Woodward) + +* SOLR-8564: Fix Embedded ZooKeeper to use <solrHome>/zoo_data for it's data directory + +* SOLR-8371: Try and prevent too many recovery requests from stacking up and clean up some faulty + cancel recovery logic. (Mark Miller) + +* SOLR-8582 : memory leak in JsonRecordReader affecting /update/json/docs. Large payloads + cause OOM (noble, shalin) + +* SOLR-8605: Regular expression queries starting with escaped forward slash caused + an exception. (Scott Blum, yonik) + +* SOLR-8607: The Schema API refuses to add new fields that match existing dynamic fields. + (Jan Høydahl, Steve Rowe) + +* SOLR-8575: Fix HDFSLogReader replay status numbers, a performance bug where we can reopen + FSDataInputStream much too often, and an hdfs tlog data integrity bug. + (Mark Miller, Patrick Dvorack, yonik) + +* SOLR-8651: The commitWithin parameter is not passed on for deleteById in UpdateRequest in + distributed queries (Jessica Cheng Mallet via Erick Erickson) + +* SOLR-8551: Make collection deletion more robust. (Mark Miller) + +Optimizations +---------------------- + +* SOLR-8501: Specify the entity request size when known in HttpSolrClient. (Mark Miller) + +* SOLR-8615: Just like creating cores, we should use multiple threads when closing cores. + (Mark Miller) + +* SOLR-7281: Add an overseer action to publish an entire node as 'down'. (Mark Miller, shalin) + +* SOLR-8669: Non binary responses use chunked encoding because we flush the outputstream early. + (Mark Miller) + +Other Changes +---------------------- + +* LUCENE-6900: Added test for score ordered grouping, and refactored TopGroupsResultTransformer. + (David Smiley) + +* SOLR-8336: CoreDescriptor now takes a Path for its instance directory, rather + than a String (Alan Woodward) + +* SOLR-8351: Improve HdfsDirectory toString representation + (Mike Drob via Gregory Chanan) + +* SOLR-8321: add a (SolrQueryRequest free) SortSpecParsing.parseSortSpec variant + (Christine Poerschke) + +* SOLR-8338: in OverseerTest replace strings such as "collection1" and "state" with variable + or enum equivalent (Christine Poerschke) + +* SOLR-8333: Several API tweaks so that public APIs were no longer refering to private classes + (ehatcher, Shawn Heisey, hossman) + +* SOLR-8357: UpdateLog.RecentUpdates now implements Closeable (Alan Woodward) + +* SOLR-8339: Refactor SolrDocument and SolrInputDocument to have a common base abstract class + called SolrDocumentBase. Deprecated methods toSolrInputDocument and toSolrDocument in ClientUtils. + (Ishan Chattopadhyaya via shalin) + +* SOLR-8353: Support regex for skipping license checksums (Gregory Chanan) + +* SOLR-8313: SimpleQueryParser doesn't use MultiTermAnalysis for Fuzzy Queries (Tom Hill via Erick Erickson) + +* SOLR-8359: Restrict child classes from using parent logger's state + (Jason Gerlowski, Mike Drob, Anshum Gupta) + +* SOLR-8131: All example config sets now explicitly use the ManagedIndexSchemaFactory + instead of ClassicIndexSchemaFactory. This means that the Schema APIs ( /<collection>/schema ) + are enabled by default and the schema is mutable. The schema file will be called managed-schema + (Uwe Schindler, shalin, Varun Thacker) + +* SOLR-8381: Cleanup data_driven managed-schema and solrconfig.xml files. Commented out copyFields are removed + and solrconfig.xml doesn't refer to field which are not defined. (Varun Thacker) + +* SOLR-7774: revise BasicDistributedZkTest.test logic w.r.t. 'commitWithin did not work on some nodes' + (Christine Poerschke) + +* SOLR-8360: simplify ExternalFileField.getValueSource implementation (Christine Poerschke) + +* SOLR-8387: All example configs shipped with Solr explicitly use ManagedIndexSchemaFactory, the schema file will + be called managed-schema instead of schema.xml . It is not advised to use hand edit the managed-schema. You should + use the schema APIs instead ( /<collection>/schema ) . If you do not want this behaviour in the example configs, + before you start solr rename managed-schema to schema.xml and change the schemaFactory in solrconfig.xml file + to explicitly use ClassicIndexSchemaFactory instead : <schemaFactory class="ClassicIndexSchemaFactory"/> + (Varun Thacker) + +* SOLR-8305: replace LatLonType.getValueSource's QParser use (Christine Poerschke) + +* SOLR-8388: factor out response/TestSolrQueryResponse.java from servlet/ResponseHeaderTest.java + more TestSolrQueryResponse.java tests; add SolrReturnFields.toString method, ReturnFieldsTest.testToString test; + (Christine Poerschke) + +* SOLR-8383: SolrCore.java + QParserPlugin.java container initialCapacity tweaks + (Christine Poerschke, Mike Drob) + +* LUCENE-6925: add RandomForceMergePolicy class in test-framework (Christine Poerschke) + +* SOLR-8404: tweak SolrQueryResponse.getToLogAsString, add TestSolrQueryResponse.testToLog (Christine Poerschke) + +* SOLR-8352: randomise unload order in UnloadDistributedZkTest.testUnloadShardAndCollection (Christine Poerschke) + +* SOLR-8414: AbstractDistribZkTestBase.verifyReplicaStatus could throw NPE (Christine Poerschke) + +* SOLR-8410: Add all read paths to 'read' permission in RuleBasedAuthorizationPlugin (noble) + +* SOLR-8279: Add a new test fault injection approach and a new SolrCloud test that stops and starts the cluster + while indexing data and with random faults. (Mark Miller) + +* SOLR-8419: TermVectorComponent for distributed search now requires a uniqueKey in the schema. Also, it no longer + returns "uniqueKeyField" in the response. (David Smiley) + +* SOLR-8317: add & use responseHeader and response accessors to SolrQueryResponse. (Christine Poerschke) + +* SOLR-8452: replace "partialResults" occurrences with SolrQueryResponse.RESPONSE_HEADER_PARTIAL_RESULTS_KEY + (Christine Poerschke) + +* SOLR-8454: ZkStateReader logging improvements and cleanup of dead code (Shai Erera, Anshum Gupta) + +* SOLR-8455: RecovertStrategy logging improvements and sleep-between-recovery-attempts bug fix. + (Shai Erera) + +* SOLR-8481: TestSearchPerf no longer needs to duplicate SolrIndexSearcher.(NO_CHECK_QCACHE|NO_CHECK_FILTERCACHE) + (Christine Poerschke) + +* SOLR-8486: No longer require jar/unzip for bin/solr (Steven E. Harris, janhoy) + +* SOLR-8483: relocate 'IMPORTANT NOTE' in open-exchange-rates.json test-file to avoid + OpenExchangeRatesOrgProvider.java warnings (Christine Poerschke) + +* SOLR-8489: TestMiniSolrCloudCluster.createCollection to support extra & alternative collectionProperties + (Christine Poerschke) + +* SOLR-8482: add & use QueryCommand.[gs]etTerminateEarly accessors. (Christine Poerschke) + +* SOLR-8498: Improve error message when a large value is stored in an indexed string field. (shalin) + +* SOLR-8484: refactor update/SolrIndexConfig.LOCK_TYPE_* into core/DirectoryFactory.LOCK_TYPE_* + (Christine Poerschke) + +* SOLR-8504: (IndexSchema|SolrIndexConfig)Test: private static finals for + solrconfig.xml and schema.xml String literals. (Christine Poerschke) + +* SOLR-8505: core/DirectoryFactory.LOCK_TYPE_HDFS - add & use it instead of String literals + (Christine Poerschke) + +* SOLR-7042: bin/post now uses /update/json/docs for application/json content types, including support for + .jsonl (JSON Lines) files. (Erik Hatcher and shalin) + +* SOLR-8476: Refactor and cleanup CoreAdminHandler (noble, Varun Thacker) + +* SOLR-8535: Support forcing define-lucene-javadoc-url to be local (Gregory Chanan) + +* SOLR-8549: Solr start script checks for cores which have failed to load as well before attempting to + create a core with the same name (Varun Thacker) + +* SOLR-8555: SearchGroupShardResponseProcessor (initialCapacity) tweaks (Christine Poerschke) + +* LUCENE-6978: Refactor several code places that lookup locales + by string name to use BCP47 locale tag instead. LuceneTestCase + now also prints locales on failing tests this way. In addition, + several places in Solr now additionally support BCP47 in config + files. (Uwe Schindler, Robert Muir) + +* SOLR-7907: Remove CLUSTERSTATUS related exclusivity checks while running commands in the Overseer because the + CLUSTERSTATUS request is served by the individual nodes itself and not via the Overseer node (Varun Thacker) + +* SOLR-8566: various initialCapacity tweaks (Fix Versions: trunk 5.5) + (Christine Poerschke) + +* SOLR-8565: add & use CommonParams.(ROWS|START)_DEFAULT constants (Christine Poerschke) + +* SOLR-8595: Use BinaryRequestWriter by default in HttpSolrClient and ConcurrentUpdateSolrClient. (shalin) + +* SOLR-8597: add default, no-op QParserPlugin.init(NamedList) method (Christine Poerschke) + +* SOLR-7968: Make QueryComponent more extensible. (Markus Jelsma via David Smiley) + +* SOLR-8600: add & use ReRankQParserPlugin parameter [default] constants, + changed ReRankQuery.toString to use StringBuilder. (Christine Poerschke) + +* SOLR-8308: Core gets inaccessible after RENAME operation with special characters + (Erik Hatcher, Erick Erickson) + +* SOLR-3141: Warn in logs when expensive optimize calls are made (yonik, janhoy) + +================== 5.4.1 ================== + +Bug Fixes +---------------------- + +* SOLR-8460: /analysis/field could throw exceptions for custom attributes. (David Smiley, Uwe Schindler) + +* SOLR-8373: KerberosPlugin: Using multiple nodes on same machine leads clients to + fetch TGT for every request (Ishan Chattopadhyaya via noble) + +* SOLR-8059: &debug=results for distributed search when distrib.singlePass (sometimes activated + automatically) could result in an NPE. (David Smiley, Markus Jelsma) + +* SOLR-8422: When authentication enabled, requests fail if sent to a node that doesn't host + the collection (noble) + +* SOLR-7462: AIOOBE in RecordingJSONParser (Scott Dawson, noble) + +* SOLR-8496: SolrIndexSearcher.getDocSet(List<Query>) incorrectly included deleted documents + when all of the queries were uncached (or there was no filter cache). This caused + multi-select faceting (including the JSON Facet API) to include deleted doc counts + when the remaining non-excluded filters were all uncached. This bug was first introduced in 5.3.0 + (Andreas Müller, Vasiliy Bout, Erick Erickson, Shawn Heisey, Hossman, yonik) + +* SOLR-8418: Adapt to changes in LUCENE-6590 for use of boosts with MLTHandler and + Simple/CloudMLTQParser (Jens Wille, Ramkumar Aiyengar) + +New Features +---------------------- + +* SOLR-8470: Make TTL of PKIAuthenticationPlugin's tokens configurable through a system property + (pkiauth.ttl) (noble) + +================== 5.4.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release + +Versions of Major Components +--------------------- +Apache Tika 1.7 +Carrot2 3.10.4 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.2.13.v20150730 + +Upgrading from Solr 5.3 +----------------------- + +* DefaultSimilarityFactory has been renamed to ClassicSimilarityFactory to match the underlying rename of + DefaultSimilarity to ClassicSimilarity and the (eventual) move away from using it as a default. + If you currently have DefaultSimilarityFactory explicitly referenced in your schema.xml, you will now get + a warning urging you to edit your config to use the functionally identical ClassicSimilarityFactory. + DefaultSimilarityFactory will be removed completely in Solr 6. See SOLR-8239 for more details. + +* SOLR-7859: The following APIs are now deprecated: + - SolrCore.getStartTime: Use SolrCore.getStartTimeStamp instead. + - SolrIndexSearcher.getOpenTime: Use SolrIndexSearcher.getOpenTimeStamp instead. + +* SOLR-8307: EmptyEntityResolver was moved from core to solrj, and moved from the org.apache.solr.util + package to org.apache.solr.common. If you are using this class, you will need to adjust the import package. + +* Logger declarations in most source files have changed to code that + no longer needs to explicitly state the class name. This fixes situations + where a logger for a different class was incorrectly used. See SOLR-8324 + and its sub-issues for details. + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-5756: A utility Collection API to move a collection from shared clusterstate.json (stateFormat=1, + default until 4.x) to the per-collection state.json stored in ZooKeeper (stateFormat=2, + default since 5.0) seamlessly without any application down-time. + Example: + http://localhost:8983/solr/admin/collections?action=MIGRATESTATEFORMAT&collection=<collection_name> + (Noble Paul, Scott Blum, shalin) + +* SOLR-7219: filterCache access added to the solr query syntax. + Example: description:HDTV OR filter(+promotion:tv +promotion_date:[NOW/DAY TO NOW/DAY+7DAY]) + (yonik) + +* SOLR-7775: Allow fromIndex parameter to ScoreJoinQParserPlugin {!join score=.. fromIndex=..}.. + to refer to a single-sharded collection that has a replica on all nodes where there is a + replica in the to index (Andrei Beliakov via Mikhail Khludnev) + +* SOLR-7961: Print Solr's version with command bin/solr version (janhoy) + +* SOLR-7789: Introduce a ConfigSet management API (Gregory Chanan) + +* SOLR-4316: Add a collections dropdown to angular admin UI (Upayavira, Shalin Shekhar Mangar) + +* SOLR-7915: Provide pluggable context tool support for VelocityResponseWriter (Erik Hatcher) + +* LUCENE-6795: SystemInfoHandler was improved to also show detailed operating + system statistics on IBM J9 virtual machines. It also no longer fails on Java 9 + with Jigsaw module system. (Uwe Schindler) + +* SOLR-8053: Basic auth support in SolrJ (noble) + +* SOLR-7995: Add a LIST command to ConfigSets API (Gregory Chanan) + +* SOLR-4388: In Angular UI, add a Collections UI when in cloud mode (Upayavira) + +* SOLR-7858, SOLR-8199: Add links between original and new Admin UIs (Upayavira) + +* SOLR-7888: Analyzing suggesters can now filter suggestions by a context field (Arcadius Ahouansou, janhoy) + +* SOLR-8217: JSON Facet API: add "method" param to terms/field facets to give an execution + hint for what method should be used to facet. (yonik) + +* SOLR-8113: CloneFieldUpdateProcessorFactory now supports choosing a "dest" field name based on a regex + pattern and replacement init options. (Gus Heck, hossman) + +* SOLR-8139: Create/delete fields/dynamic fields/copy fields via schema tab on Angular UI + +* SOLR-8166: Introduce possibility to configure ParseContext in + ExtractingRequestHandler/ExtractingDocumentLoader (Andriy Binetsky + via Uwe Schindler) + +* SOLR-7569: A collection API to force elect a leader, called FORCELEADER, when all replicas in a shard are down + (Ishan Chattopadhyaya, Mark Miller, shalin, noble) + +* SOLR-6168: Add a 'sort' local param to the collapse QParser to support using complex sort options + to select the representitive doc for each collapsed group. (Umesh Prasad, hossman) + +* SOLR-8329: SchemaSimilarityFactory now supports a 'defaultSimFromFieldType' init option for using + a fieldType name to identify which Similarity to use as a default. (hossman) + +* SOLR-7912: Add boost support, and also exclude the queried document in MoreLikeThis QParser + (Jens Wille via Anshum Gupta) + +Bug Fixes +---------------------- + +* SOLR-7859: Fix usage of currentTimeMillis instead of nanoTime in multiple places, + whitelist valid uses of currentTimeMillis (Ramkumar Aiyengar) + +* SOLR-7836: Possible deadlock when closing refcounted index writers. + (Jessica Cheng Mallet, Erick Erickson, Mark Miller, yonik) + +* SOLR-7869: Overseer does not handle BadVersionException correctly and, in some cases, + can go into an infinite loop if cluster state in ZooKeeper is modified externally. + (Scott Blum, shalin) + +* SOLR-7920: Resolve XSS issue in Admin UI Schema Browser (David Chiu via Upayavira) + +* SOLR-7935: Fix very rare race condition that can cause an update to fail + via NullPointerException during a core reload. (yonik) + +* SOLR-7941: multivalued params are concatenated when using config API (noble) + +* SOLR-7956: There are interrupts on shutdown in places that can cause ChannelAlreadyClosed + exceptions which prevents proper closing of transaction logs, interfere with the IndexWriter, + the hdfs client and other things. (Mark Miller, Scott Blum) + +* SOLR-7954: Fixed an integer overflow bug in the HyperLogLog code used by the 'cardinality' option + of stats.field to prevent ArrayIndexOutOfBoundsException in a distributed search when a large precision + is selected and a large number of values exist in each shard (hossman) + +* SOLR-7844: Zookeeper session expiry during shard leader election can cause multiple leaders. + (Mike Roberts, Mark Miller, Jessica Cheng) + +* SOLR-7984: wrong and misleading error message 'no default request handler is registered' (noble, hossman) + +* SOLR-8001: Fixed bugs in field(foo,min) and field(foo,max) when some docs have no values + (David Smiley, hossman) + +* SOLR-7819: ZK connection loss or session timeout do not stall indexing threads anymore. All activity + related to leader initiated recovery is performed by a dedicated LIR thread in the background. + (Ramkumar Aiyengar, shalin) + +* SOLR-7746: Ping requests stopped working with distrib=true in Solr 5.2.1. + (Alexey Serba, Michael Sun via Gregory Chanan) + +* SOLR-6547: ClassCastException in SolrResponseBase.getQTime on update response from CloudSolrClient + when parallelUpdates is enabled (default) and multiple docs are sent as a single update. + (kevin, hossman, shalin) + +* SOLR-8058: Fix the exclusion filter so that collections that start with js, css, img, tpl + can be accessed. (Upayavira, Steve Rowe, Anshum Gupta) + +* SOLR-8069: Ensure that only the valid ZooKeeper registered leader can put a replica into Leader + Initiated Recovery. (Mark Miller, Jessica Cheng, Anshum Gupta) + +* SOLR-8077: Replication can still cause index corruption. (Mark Miller) + +* SOLR-8104: Config API does not work for spellchecker (noble) + +* SOLR-8095: Allow disabling HDFS Locality Metrics and disable by default as it may have performance + implications on rapidly changing indexes. (Mike Drob via Mark Miller) + +* SOLR-8085: Fix a variety of issues that can result in replicas getting out of sync. (yonik, Mark Miller) + +* SOLR-8094: HdfsUpdateLog should not replay buffered documents as a replacement to dropping them. + (Mark Miller) + +* SOLR-8075: Leader Initiated Recovery should not stop a leader that participated in an election with all + of it's replicas from becoming a valid leader. (Mark Miller) + +* SOLR-8072: Rebalance leaders feature does not set CloudDescriptor#isLeader to false when bumping leaders. + (Mark Miller) + +* SOLR-7666: Many small fixes to Angular UI (Upayavira, Alexandre Rafalovitch) + +* SOLR-7967: AddSchemaFieldsUpdateProcessorFactory does not check if the ConfigSet is immutable (Gregory Chanan) + +* SOLR-6188: Skip the automatic loading of resources in the "lib" subdirectory + by SolrResourceLoader, but only if we are loading resources from the solr + home directory. Fixes the inability to use ICU analysis components with a + "solr." prefix on the classname. (Shawn Heisey) + +* SOLR-8130: Solr's hdfs safe mode detection does not catch all cases of being in safe mode. + (Mark Miller, Mike Drob) + +* SOLR-8128: Set v.locale specified locale for all LocaleConfig extending VelocityResponseWriter tools. + (Erik Hatcher) + +* SOLR-8152: Overseer Task Processor/Queue can miss responses, leading to timeouts. + (Gregory Chanan) + +* SOLR-8107: bin/solr -f should use exec to start the JVM (Martijn Koster via Timothy Potter) + +* SOLR-8050: Partial update on document with multivalued date field fails to parse date and can + also fail to remove dates in some cases. (Burkhard Buelte, Luc Vanlerberghe, shalin) + +* SOLR-8167: Authorization framework does not work with POST params (noble) + +* SOLR-8162: JmxMonitoredMap#clear triggers a query on all the MBeans thus generating lots of warnings. + (Marius Dumitru Florea, shalin) + +* SOLR-7843: DataImportHandler's delta imports leak memory because the delta keys are kept in memory + and not cleared after the process is finished. (Pablo Lozano via shalin) + +* SOLR-8189: eTag calculation during HTTP Cache Validation uses unsynchronized WeakHashMap causing + threads to be stuck in runnable state. (shalin) + +* SOLR-7993: Raw json output for fields stopped working in 5.3.0 when requested fields do not include + the unique key field name. (Bill Bell, Ryan McKinley via shalin) + +* SOLR-8192: JSON Facet API allBuckets:true did not work correctly when faceting + on a multi-valued field with sub-facets / facet functions. (yonik) + +* SOLR-8206: JSON Facet API limit:0 did not always work correctly. (yonik) + +* SOLR-8126: update-<component-name> does not work if the component is only + present in solrconfig.xml (noble) + +* SOLR-8203: Stop processing updates more quickly on node shutdown. When a node + is shut down, streaming updates would continue, but new update requests would + be aborted. This can cause big update reorders that can cause replicas to + get out of sync. (Mark Miller, yonik) + +* SOLR-6406: ConcurrentUpdateSolrClient hang in blockUntilFinished. If updates are still + flowing and shutdown is called on the executor service used by ConcurrentUpdateSolrClient, + a race condition can cause that client to hang in blockUntilFinished. + (Mark Miller, yonik) + + +* SOLR-8215: Only active replicas should handle incoming requests against a collection (Varun Thacker) + +* SOLR-8223: Avoid accidentally swallowing OutOfMemoryError (in LeaderInitiatedRecoveryThread.java + or CoreContainer.java) (Mike Drob via Christine Poerschke) + +* SOLR-8255: MiniSolrCloudCluster needs to use a thread-safe list to keep track + of its child nodes (Alan Woodward) + +* SOLR-8254: HttpSolrCore.getCoreByCollection() can throw NPE (Alan Woodward, + Mark Miller) + +* SOLR-8262: Comment out /stream handler from sample solrconfig.xml's for security reasons + (Joel Bernstein) + +* SOLR-7989: After a new leader is elected it should change it's state to ACTIVE even + if the last published state is something else if it has already registered with ZK. + (Ishan Chattopadhyaya, Mark Miller via noble) + +* SOLR-8287: TrieDoubleField and TrieLongField now override toNativeType + (Ishan Chattopadhyaya via Christine Poerschke) + +* SOLR-8284: JSON Facet API - fix NPEs when short form "sort:index" or "sort:count" + are used. (Michael Sun via yonik) + +* SOLR-8295: Fix NPE in collapse QParser when collapse field is missing from all docs in a segment (hossman) + +* SOLR-8280: Fixed bug in SimilarityFactory initialization that prevented SolrCoreAware factories -- such + as SchemaSimilarityFactory -- from functioning properly with managed schema features. (hossman) + +* SOLR-5971: Fix error 'Illegal character in query' when proxying request. + (Uwe Schindler, Ishan Chattopadhyaya, Eric Bus) + +* SOLR-8307: Fix XXE vulnerability in MBeansHandler "diff" feature (Erik Hatcher) + +* SOLR-8073: Solr fails to start on Windows with obscure errors when using relative path. + (Alexandre Rafalovitch, Ishan Chattopadhyaya via shalin) + +* SOLR-7169: bin/solr status should return exit code 3, not 0 if Solr is not running + (Dominik Siebel via Timothy Potter) + +* SOLR-8341: Fix JSON Facet API excludeTags when specified in the + form of domain:{excludeTags:mytag} (yonik) + +* SOLR-8326: If BasicAuth enabled, inter node requests fail after node restart (noble, Anshum Gupta) + +* SOLR-8340: Fixed NullPointerException in HighlightComponent. + (zengjie via Christine Poerschke) + +* SOLR-8355: update permissions were failing node recovery (noble , Anshum Gupta) + +Optimizations +---------------------- + +* SOLR-7918: Filter (DocSet) production from term queries has been optimized and + is anywhere from 20% to over 100% faster and produces less garbage on average. + (yonik) + +* SOLR-6760: New optimized DistributedQueue implementation for overseer increases + message processing performance by ~470%. + (Noble Paul, Scott Blum, shalin) + +* SOLR-6629: Watch /collections zk node on all nodes so that cluster state updates + are more efficient especially when cluster has a mix of collections in stateFormat=1 + and stateFormat=2. (Scott Blum, shalin) + +* SOLR-7971: Reduce memory allocated by JavaBinCodec to encode small strings by an amount + equal to the string.length(). JavaBinCodec now uses a double pass approach to write strings + larger than 64KB to avoid allocating buffer memory equal to string's UTF8 size. + (yonik, Steve Rowe, Mikhail Khludnev, Noble Paul, shalin) + +* SOLR-7983: Utils.toUTF8 uses larger buffer than necessary for holding UTF8 data. (shalin) + +* SOLR-8222: JSON Facet API optimization to faceting by count on docvalue fields (or indexed fields + with method=dv) when there are multiple hits expected for enoug buckets. For example, this + more than doubled the performance of faceting 5M documents over a field with 1M unique values. + (yonik) + +* SOLR-8288: DistributedUpdateProcessor#doFinish should explicitly check and ensure it + does not try to put itself into LIR. (Mark Miller) + +Other Changes +---------------------- + +* SOLR-8294: Cleanup solrconfig.xmls under solr/example/example-DIH/solr (removed + obsolete clustering handler sections). (Dawid Weiss) + +* SOLR-7969: Unavailable clustering engines should not fail the core. (Dawid Weiss) + +* SOLR-7790, SOLR-7791: Update Carrot2 clustering component to + version 3.10.4. Upgrade HPPC library to version 0.7.1. (Dawid Weiss) + +* SOLR-7831: Start Scripts: Allow a configurable stack size [-Xss] (Steve Davids via Mark Miller) + +* SOLR-7870: Write a test which asserts that requests to stateFormat=2 collection succeed on a node + even after all local replicas of that collection have been removed. + (Scott Blum via shalin) + +* SOLR-7902: Split out use of child timers from RTimer to a sub-class + (Ramkumar Aiyengar) + +* SOLR-7943: Upgrade Jetty to 9.2.13.v20150730. (Bill Bell, shalin) + +* SOLR-7007: DistributedUpdateProcessor now logs replay flag as boolean instead of int + (Mike Drob via Christine Poerschke) + +* SOLR-7960: Start scripts now gives generic help for bin/solr -h and bin/solr --help (janhoy) + +* SOLR-7970: Factor out a SearchGroupsFieldCommandResult class. + (Christine Poerschke) + +* SOLR-7942: Previously removed unlockOnStartup option (LUCENE-6508) now logs warning if configured, + will be an error in 6.0. Also improved error msg if an index is locked on startup (hossman) + +* SOLR-7979: Fix two typos (in a CoreAdminHandler log message and a TestCloudPivotFacet comment). + (Mike Drob via Christine Poerschke) + +* SOLR-7966: Solr Admin UI Solr now sets the HTTP header X-Frame-Options to DENY + to avoid clickjacking. (yonik) + +* SOLR-7999: SolrRequestParser tests no longer depend on external URLs + that may fail to work. (Uwe Schindler) + +* SOLR-8034: Leader no longer puts replicas in recovery in case of a failed update, when minRF + isn't achieved. (Jessica Cheng, Timothy Potter, Anshum Gupta) + +* SOLR-8066: SolrCore.checkStale method doesn't restore interrupt status. (shalin) + +* SOLR-8068: Throw a SolrException if the core container has initialization errors or is + shutting down (Ishan Chattopadhyaya, Noble Paul, Anshum Gupta) + +* SOLR-8083: Convert the ZookeeperInfoServlet to a handler at /admin/zookeeper (noble) + +* SOLR-8025: remove unnecessary ResponseBuilder.getQueryCommand() calls (Christine Poerschke) + +* SOLR-8150: Fix build failure due to too much output from QueryResponseTest (janhoy) + +* SOLR-8151: OverseerCollectionMessageHandler was logging info data as WARN + (Alan Woodward) + +* SOLR-8116: SearchGroupsResultTransformer tweaks (String literals, list/map initialCapacity) + (Christine Poerschke) + +* SOLR-8114: in Grouping.java rename groupSort and sort to withinGroupSort and groupSort + (Christine Poerschke) + +* SOLR-8074: LoadAdminUIServlet directly references admin.html (Mark Miller, Upayavira) + +* SOLR-8195: IndexFetcher download trace now includes bytes-downloaded[-per-second] + (Christine Poerschke) + +* SOLR-4854: Add a test to assert that [elevated] DocTransfer works correctly with javabin + response format. (Ray, shalin) + +* SOLR-8196: TestMiniSolrCloudCluster.testStopAllStartAll case plus necessary + MiniSolrCloudCluster tweak (Christine Poerschke) + +* SOLR-8221: MiniSolrCloudCluster should create subdirectories for its nodes + (Alan Woodward) + +* SOLR-8218: DistributedUpdateProcessor (initialCapacity) tweaks + (Christine Poerschke) + +* SOLR-8147: contrib/analytics FieldFacetAccumulator now throws IOException instead of SolrException + (Scott Stults via Christine Poerschke) + +* SOLR-8239: Added ClassicSimilarityFactory, marked DefaultSimilarityFactory as deprecated. (hossman) + +* SOLR-8253: AbstractDistribZkTestBase can sometimes fail to shut down its + ZKServer (Alan Woodward) + +* SOLR-8260: Use NIO2 APIs in core discovery (Alan Woodward) + +* SOLR-8259: Deprecate JettySolrRunner.getDispatchFilter(), add + .getSolrDispatchFilter() and .getCoreContainer() (Alan Woodward) + +* SOLR-8278: Use NIO2 APIs in ConfigSetService (Alan Woodward) + +* SOLR-8286: Remove instances of solr.hdfs.blockcache.write.enabled from tests + and docs (Gregory Chanan) + +* SOLR-8269: Upgrade commons-collections to 3.2.2. This fixes a known serialization vulnerability (janhoy) + +* SOLR-8246: Fix SolrCLI to clean the config directory in case creating a core failed. + (Jason Gerlowski via Shai Erera) + +* SOLR-8290: remove SchemaField.checkFieldCacheSource's unused QParser argument (Christine Poerschke) + +* SOLR-8300: Use constants for the /overseer_elect znode (Varun Thacker) + +* SOLR-8283: factor out StrParser from QueryParsing.StrParser and SortSpecParsing[Test] + from QueryParsing[Test] (Christine Poerschke) + +* SOLR-8298: small preferLocalShards implementation refactor (Christine Poerschke) + +* SOLR-8315: Removed default core checks in the dispatch filter since we don't have a default + core anymore (Varun Thacker) + +* SOLR-8302: SolrResourceLoader now takes a Path as its instance directory (Alan + Woodward, Shawn Heisey) + +* SOLR-8303: CustomBufferedIndexInput now includes resource description when + throwing EOFException. (Mike Drob via Uwe Schindler) + +* SOLR-8194: Improve error reporting for null documents in UpdateRequest (Markus + Jelsma, Alan Woodward) + +* SOLR-8277: (Search|Top)GroupsFieldCommand tweaks (Christine Poerschke) + +* SOLR-8299: ConfigSet DELETE operation no longer allows deletion of config sets that + are currently in use by other collections (Anshum Gupta) + +* SOLR-8101: Improve Linux service installation script (Sergey Urushkin via Timothy Potter) + +* SOLR-8180: jcl-over-slf4j should have officially been a SolrJ dependency; it now is. + (David Smiley, Kevin Risden) + +* SOLR-8330: Standardize and fix logger creation and usage so that they aren't shared + across source files.(Jason Gerlowski, Uwe Schindler, Anshum Gupta) + +* SOLR-8363: Fix check-example-lucene-match-version Ant task and addVersion.py script to + check and update luceneMatchVersion under solr/example/ configs as well logic. (Varun Thacker) + +================== 5.3.2 ================== + +Bug Fixes +---------------------- + +* SOLR-8460: /analysis/field could throw exceptions for custom attributes. (David Smiley, Uwe Schindler) + +* SOLR-8373: KerberosPlugin: Using multiple nodes on same machine leads clients to + fetch TGT for every request (Ishan Chattopadhyaya via noble) + +* SOLR-8340: Fixed NullPointerException in HighlightComponent. (zengjie via Christine Poerschke) + +* SOLR-8059: &debug=results for distributed search when distrib.singlePass (sometimes activated + automatically) could result in an NPE. (David Smiley, Markus Jelsma) + +* SOLR-8167: Authorization framework does not work with POST params (noble) + +* SOLR-8355: update permissions were failing node recovery (noble , Anshum Gupta) + +* SOLR-8326: If BasicAuth enabled, inter node requests fail after node restart (noble, Anshum Gupta) + +* SOLR-8269: Upgrade commons-collections to 3.2.2. This fixes a known serialization vulnerability (janhoy) + +* SOLR-8422: When authentication enabled, requests fail if sent to a node that doesn't host + the collection (noble) + +* SOLR-8496: SolrIndexSearcher.getDocSet(List<Query>) incorrectly included deleted documents + when all of the queries were uncached (or there was no filter cache). This caused + multi-select faceting (including the JSON Facet API) to include deleted doc counts + when the remaining non-excluded filters were all uncached. This bug was first introduced in 5.3.0 + (Andreas Müller, Vasiliy Bout, Erick Erickson, Shawn Heisey, Hossman, yonik) + + +================== 5.3.1 ================== + +Bug Fixes +---------------------- + +* SOLR-7949: Resolve XSS issue in Admin UI stats page (David Chiu via janhoy) + +* SOLR-8000: security.json is not loaded on server start (noble) + +* SOLR-8004: RuleBasedAuthorization plugin does not work for the collection-admin-edit permission (noble) + +* SOLR-7972: Fix VelocityResponseWriter template encoding issue. + Templates must be UTF-8 encoded. (Erik Hatcher) + +* SOLR-7929: SimplePostTool (also bin/post) -filetypes "*" now works properly in 'web' mode (Erik Hatcher) + +* SOLR-7978: Fixed example/files update-script.js to be Java 7 and 8 compatible. (Erik Hatcher) + +* SOLR-7988: SolrJ could not make requests to handlers with '/admin/' prefix (noble , ludovic Boutros) + +* SOLR-7990: Use of timeAllowed can cause incomplete filters to be cached and incorrect + results to be returned on subsequent requests. (Erick Erickson, yonik) + +* SOLR-8041: Fix VelocityResponseWriter's $resource.get(key,baseName,locale) to use specified locale. + (Erik Hatcher) + +================== 5.3.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release + +Versions of Major Components +--------------------- +Apache Tika 1.7 +Carrot2 3.9.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.2.11.v20150529 + +Upgrading from Solr 5.2 +----------------------- + +* SolrJ's CollectionAdminRequest class is now marked as abstract. Use one of its concrete + sub-classes instead. + +* Solr no longer supports forcefully unlocking an index. + This is no longer supported by the underlying Lucene locking + framework. The setting in solrconfig.xml has no effect anymore. + Background: If you use native lock factory, unlocking should + not be needed, because the locks are cleared after process + shutdown automatically by the operating system. If you are + using simple lock factory (not recommended) or hdfs lock + factory, you may need to manually unlock by deleting the lock + file from filesystem / HDFS. + +* The zkCredientialsProvider element in solrcloud section of solr.xml is now deprecated. + Use the correct spelling (zkCredentialsProvider) instead. + +* class TransformerWithContext is deprecated . Use DocTransformer directly + +* The "name" parameter in ADDREPLICA Collections API call has be deprecated. One cannot specify + the core name for a replica. See SOLR-7499 for more info. + +* The ShardHandler interface has changed. The interface used to provide a + `checkDistributed` function which doubled up in purpose to determine if the + request is distributed, and to prepare for distributed requests. This unfortunately + meant that the object had to be instantiated even when the request is not + distributed. The task of initially determining if the request is distributed + is now done by SearchHandler using the distrib/shards parameters, and a + ShardHandler object is created only if the request is distributed. The interface + now has a `prepDistributed` function instead of the `checkDistributed` function, + which can then be used to prepare for the distributed request. Users with custom + ShardHandler implementations would need to modify their code to this effect. + +* The system property "solr.solrxml.location" is not supported any more. Now, solr.xml is first + looked up in zookeeper, and if not found, fallback to SOLR_HOME. See SOLR-7735 for more info. + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-7724: SolrJ now supports parsing the output of the clustering component. + (Alessandro Benedetti via Dawid Weiss) + +* SOLR-7389: Expose znodeVersion property for each of the collections returned for the clusterstatus + operation in the collections API (Marius Grama via shalin) + +* SOLR-7622: A DocTransformer can now request fields from the SolrIndexSearcher that are not + necessarily returned in the file SolrDocument by returning a list of fields from + DocTransformer#getExtraRequestFields (ryan) + +* SOLR-7458: Expose HDFS Block Locality Metrics via JMX (Mike Drob via Mark Miller) + +* SOLR-7676: Faceting on nested objects / Block-join faceting with the new JSON Facet API. + Example: Assuming books with nested pages and an input domain of pages, the following + will switch the domain to books before faceting on the author field: + authors:{ type:terms, field:author, domain:{toParent:"type:book"} } + (yonik) + +* SOLR-7668: Add 'port' tag support in replica placement rules (Adam McElwee, Noble Paul) + +* SOLR-5886: Response for an async call is now stored in zk so that it can be returned by the REQUESTSTATUS API. + Also, the number of stored (failed and successful) responses are now restricted to 10,000 each as a safety net. + (Anshum Gupta) + +* SOLR-7639: MoreLikeThis QParser now supports all options provided by the MLT Handler i.e. mintf, mindf, + minwl, maxwl, maxqt, and maxntp. + +* SOLR-7182: Make the Schema-API a first class citizen of SolrJ. The new SchemaRequest and its inner + classes can be used to make requests to the Schema API. (Sven Windisch, Marius Grama via shalin) + +* SOLR-7651: New response format added wt=smile (noble) + +* SOLR-4212: SOLR-6353: Let facet queries and facet ranges hang off of pivots. Example: + facet.range={!tag=r1}price&facet.query={!tag=q1}somequery&facet.pivot={!range=r1 query=q1}category,manufacturer + (Steve Molloy, hossman, shalin) + +* SOLR-7742: Support for Immutable ConfigSets (Gregory Chanan) + +* SOLR-2522: new two argument option for the existing field() function; picks the min/max value of a + docValues field to use as a ValueSource: "field(field_name,min)" and "field(field_name,max)" (hossman) + +* SOLR-6234: Scoring for query time join (Mikhail Khludnev) + +* SOLR-5882: score local parameter for block join query parser {!parent} (Andrey Kudryavtsev, Mikhail Khludnev) + +* SOLR-7799: Added includeIndexFieldFlags (backwards compatible default is true) to /admin/luke. + When there are many fields in the index, setting this flag to false can dramatically speed up requests. (ehatcher) + +* SOLR-7769: Add bin/post -p alias for -port parameter. (ehatcher) + +* SOLR-7766: support creation of a coreless collection via createNodeSet=EMPTY (Christine Poerschke) + +* SOLR-7849: Solr-managed inter-node authentication when authentication enabled (Noble Paul) + +* SOLR-7220: Nested C-style comments in queries. (yonik) + +* SOLR-7757: Improved security framework where security components can be edited/reloaded, Solr + now watches /security.json. Components can choose to make their config editable + (Noble Paul, Anshum Gupta, Ishan Chattopadhyaya) + +* SOLR-7838: An authorizationPlugin interface where the access control rules are stored/managed in + ZooKeeper (Noble Paul, Anshum Gupta, Ishan Chattopadhyaya) + +* SOLR-7837: An AuthenticationPlugin which implements the HTTP BasicAuth protocol and stores credentials + securely in ZooKeeper (Noble Paul, Anshum Gupta,Ishan Chattopadhyaya) + + +Bug Fixes +---------------------- + +* SOLR-7361: Slow loading SolrCores should not hold up all other SolrCores that have finished loading from serving + requests. (Mark Miller, Timothy Potter, Ramkumar Aiyengar) + +* SOLR-4506: Clean-up old (unused) index directories in the background after initializing a new index; + previously, Solr would leave old index.yyyyMMddHHmmssSSS directories left behind after failed recoveries + in the data directory, which unnecessarily consumes disk space. (Mark Miller, Timothy Potter) + +* SOLR-7108: Change default query used by /admin/ping to not rely on other parameters such as query parser or + default field. (ehatcher) + +* SOLR-6835: ReRankQueryParserPlugin checks now whether the reRankQuery parameter is present and not empty. + (帅广应, Marius Grama via shalin) + +* SOLR-7566: Search requests should return the shard name that is down. (Marius Grama, shalin) + +* SOLR-7675: Add missing _root_ field to managed-schema template so that the default data driven + config set can index nested documents by default. (yonik) + +* SOLR-7635: Limit lsof port check in bin/solr to just listening ports + (Upayavira, Ramkumar Aiyengar) + +* SOLR-7091: Nested documents with unknown fields don't work in schemaless mode. + (Steve Rowe) + +* SOLR-7682: Schema API: add-copy-field should accept the maxChars parameter. (Steve Rowe) + +* SOLR-7693: Fix the bin/solr -e cloud example to work if lsof is not installed + on the local machine by waiting for 10 seconds before starting the second node. + (hossman, Timothy Potter) + +* SOLR-7689: ReRankQuery rewrite method can change the QueryResultKey causing cache misses. + (Emad Nashed, Yonik Seeley, Joel Bernstein) + +* SOLR-7697: Schema API doesn't take class or luceneMatchVersion attributes into + account for the analyzer when adding a new field type. (Marius Grama, Steve Rowe) + +* SOLR-7679: Schema API doesn't take similarity attribute into account when adding + field types. (Marius Grama, Steve Rowe) + +* SOLR-7664: Throw correct exception (RemoteSolrException) on receiving a HTTP 413. + (Ramkumar Aiyengar, Eirik Lygre) + +* SOLR-6686: facet.threads can return wrong results when using facet.prefix multiple + times on same field. (Michael Ryan, Tim Underwood via shalin) + +* SOLR-7673: Race condition in shard splitting can cause operation to hang indefinitely + or sub-shards to never become active. (shalin) + +* SOLR-7741: Add missing fields to SolrIndexerConfig.toMap + (Mike Drob, Christine Poerschke via Ramkumar Aiyengar) + +* SOLR-7748: Fix bin/solr to start on IBM J9. (Shai Erera) + +* SOLR-7143: MoreLikeThis Query parser should handle multiple field names + (Jens Wille, Anshum Gupta) + +* SOLR-7132: The Collections API ADDREPLICA command property.name is not reflected + in the clusterstate until after Solr restarts (Erick Erickson) + +* SOLR-7172: addreplica API fails with incorrect error msg "cannot create collection" + (Erick Erickson) + +* SOLR-7705: CoreAdminHandler Unload no longer handles null core name and throws NPE + instead of a bad request error. (John Call, Edward Ribeiro via shalin) + +* SOLR-7529: CoreAdminHandler Reload throws NPE on null core name instead of a bad + request error. (Jellyfrog, Edward Ribeiro via shalin) + +* SOLR-7781: JSON Facet API: Terms facet on string/text fields with sub-facets caused + a bug that resulted in filter cache lookup misses as well as the filter cache + exceeding it's configured size. (yonik) + +* SOLR-7810: map-reduce contrib script to set classpath for convenience refers to example + rather than server. (Mark Miller) + +* SOLR-7765: Hardened the behavior of TokenizerChain when null arguments are used in constructor. + This prevents NPEs in some code paths. (Konstantin Gribov, hossman) + +* SOLR-7829: Fixed a bug in distributed pivot faceting that could result in a facet.missing=true count + which was lower then the correct count if facet.sort=index and facet.pivot.mincount > 1 (hossman) + +* SOLR-7842: ZK connection loss or session expiry events should not fire config directory listeners. + (noble, shalin) + +* SOLR-6357: Allow delete documents by doing a score join query. (Mikhail Khludnev, Timothy Potter) + +* SOLR-7756: Fixed ExactStatsCache and LRUStatsCache to not throw an NPE when a term is not present on a shard. + (Varun Thacker, Anshum Gupta) + +* SOLR-7818: Fixed distributed stats to be calculated for all the query terms. Earlier the stats were calculated with + the terms that are present in the last shard of a distributed request. (Varun Thacker, Anshum Gupta) + +* SOLR-7866: VersionInfo caused an unhandled NPE when trying to determine the max value for the + version field. (Timothy Potter) + +* SOLR-7666 (and linked tickets): Many fixes to AngularJS Admin UI bringing it close to feature + parity with existing UI. (Upayavira) + +* SOLR-7908: SegmentsInfoRequestHandler gets a ref counted IndexWriter and does not properly release it. + (Mark Miller, shalin) + +* SOLR-7921: The techproducts example fails when running in a directory that contains spaces. + (Ishan Chattopadhyaya via Timothy Potter) + +* SOLR-7934: SolrCLI masks underlying cause of create collection failure. (Timothy Potter) + + +Optimizations +---------------------- + +* SOLR-7660: Avoid redundant 'exists' calls made to ZK while fetching cluster state updates. (shalin) + +* SOLR-7714: Reduce SearchHandler's use of ShardHandler objects across shards in a search, + from one for each shard and the federator, to just one for the federator. + (Christine Poerschke via Ramkumar Aiyengar) + +* SOLR-7751: Minor optimizations to QueryComponent.process (reduce eager instantiations, + cache method calls) (Christine Poerschke via Ramkumar Aiyengar) + +* SOLR-7455: Terms facets with the JSON Facet API now defer calculating non-sorting stats + until a second phase, after the top N facets are found. This improves performance + proportional to the number of non-sorting statistics being calculated in addition to + the number of buckets and domain documents. + For Example: The facet request {type:terms, field:field1, facet:{x:"unique(field2)"}} + saw a 7x improvement when field1 and 1M unique terms and field2 had 1000 unique terms. + (yonik) + +* SOLR-7840: ZkStateReader.updateClusterState fetches watched collections twice from ZK. (shalin) + +* SOLR-7875: Speedup SolrQueryTimeoutImpl. Avoid setting a timeout time when timeAllowed + parameter is not set. (Tomás Fernández Löbbe) + +Other Changes +---------------------- + +* SOLR-7787: Removed fastutil and java-hll dependency, integrated HyperLogLog from java-hll + into Solr core. (Dawid Weiss) + +* SOLR-7595: Allow method chaining for all CollectionAdminRequests in Solrj. (shalin) + +* SOLR-7146: MiniSolrCloudCluster based tests can fail with ZooKeeperException NoNode for /live_nodes. + (Vamsee Yarlagadda via shalin) + +* SOLR-7590: Finish and improve MDC context logging support. (Mark Miller) + +* SOLR-7599: Remove cruft from SolrCloud tests. (shalin) + +* SOLR-7636: CLUSTERSTATUS API is executed at CollectionsHandler (noble) + +* LUCENE-6508: Remove ability to forcefully unlock an index. + This is no longer supported by the underlying Lucene locking + framework. (Uwe Schindler, Mike McCandless, Robert Muir) + +* SOLR-3719: Add as-you-type "instant search" to example/files /browse. + (Esther Quansah, ehatcher) + +* SOLR-7645: Remove explicitly defined request handlers from example and test solrconfig's that are + already defined implicitly, such as /admin/ping, /admin/system, and several others. (ehatcher) + +* SOLR-7603: Fix test only bug in UpdateRequestProcessorFactoryTest (hossman) + +* SOLR-7634: Upgrade Jetty to 9.2.11.v20150529 (Bill Bell, shalin) + +* SOLR-7659: Rename releaseCommitPointAndExtendReserve in DirectoryFileStream + to extendReserveAndReleaseCommitPoint, and reverse the code to match. + (shalin, Shawn Heisey) + +* SOLR-7624: Add correct spelling (zkCredentialsProvider) as an alternative to + zkCredientialsProvider element in solrcloud section of solr.xml. + (Xu Zhang, Per Steffensen, Ramkumar Aiyengar, Mark Miller) + +* SOLR-7619: Fix SegmentsInfoRequestHandlerTest when more than one segment is created. + (Ramkumar Aiyengar, Steve Rowe) + +* SOLR-7678: Switch RTimer to use nanoTime (improves accuracy of QTime, and other times + returned by Solr handlers) (Ramkumar Aiyengar) + +* SOLR-7680: Use POST instead of GET when finding versions for mismatches with + CloudInspectUtil for tests (Ramkumar Aiyengar) + +* SOLR-7665: deprecate the class TransformerWithContext (noble) + +* SOLR-7629: Have RulesTest consider disk space limitations of where the test is + being run (Christine Poerschke via Ramkumar Aiyengar) + +* SOLR-7499: The "name" parameter in ADDREPLICA Collections API call has be deprecated. One cannot specify + the core name for a replica (Varun Thacker, noble, Erick Erickson) + +* SOLR-7711: Correct initial capacity for the list that holds the default components for the SearchHandler + (Christine Poerschke via Varun Thacker) + +* SOLR-7485: Replace shards.info occurrences with ShardParams.SHARDS_INFO + (Christine Poerschke via Ramkumar Aiyengar) + +* SOLR-7710: Replace async occurrences with CommonAdminParams.ASYNC + (Christine Poerschke, Ramkumar Aiyengar) + +* SOLR-7712: fixed test to account for aggregate floating point precision loss (hossman) + +* SOLR-7740: Fix typo bug with TestConfigOverlay (Christine Poerschke via Ramkumar Aiyengar) + +* SOLR-7750: Change TestConfig.testDefaults to cover all SolrIndexConfig fields + (Christine Poerschke via Ramkumar Aiyengar) + +* SOLR-7703: Authentication plugin is now loaded using the ResourceLoader. + (Avi Digmi via Anshum Gupta) + +* SOLR-7800: JSON Facet API: the avg() facet function now skips missing values + rather than treating them as a 0 value. The def() function can be used to + treat missing values as 0 if that is desired. + Example: facet:{ mean:"avg(def(myfield,0))" } + +* SOLR-7805: Update Kite Morphlines to 1.1.0 (Mark Miller) + +* SOLR-7803: Prevent class loading deadlock in TrieDateField; refactor date + formatting and parsing out of TrieDateField and move to static utility class + DateFormatUtil. (Markus Heiden, Uwe Schindler) + +* SOLR-7825: Forbid all usages of log4j and java.util.logging classes in Solr except + classes which are specific to logging implementations. Remove accidental usage of log4j + logger from a few places. The default log level for org.apache.zookeeper is changed from + ERROR to WARN for zkcli.{sh,cmd} only. + (Oliver Schrenk, Tim Potter, Uwe Schindler, shalin) + +* SOLR-7735: Look for solr.xml in Zookeeper by default in SolrCloud mode. If not found, it will be loaded + from $SOLR_HOME/solr.xml as before. Sysprop solr.solrxml.location is now gone. (janhoy) + +* SOLR-7227: Ship Solr with the Web application directory exploded into + server/solr-webapp, solr.war is no longer included in the distribution + bundles. (Timothy Potter, Uwe Schindler) + +* SOLR-6625: Enable registering interceptors for the calls made using HttpClient and make the + request object available at the interceptor context ( Ishan Chattopadhyay, Gregory Chanan, noble, Anshum Gupta) + +* SOLR-5022: On Java 7 raise permgen for running tests. (Uwe Schindler) + +* SOLR-7823: TestMiniSolrCloudCluster.testCollectionCreateSearchDelete async collection-creation (sometimes) + (Christine Poerschke) + +* SOLR-7854: Remove unused ZkStateReader.updateClusterState(false) method. (Scott Blum via shalin) + +* SOLR-7863: Lowercase the CLUSTERPROP command in ZkCLI for consistency, print error for unknown cmd (janhoy) + +* SOLR-7832: bin/post now allows either -url or -c, rather than requiring both. (ehatcher) + +* SOLR-7847: Implement run example logic in Java instead of OS-specific scripts in + bin/solr and bin\solr.cmd (Timothy Potter) + +* SOLR-7877: TestAuthenticationFramework.testBasics to preserve/restore the original request(Username|Password) + (Christine Poerschke) + +* SOLR-7900: example/files improvements - added language detection and faceting, added title field, relocated .js files. + (Esther Quansah and Erik Hatcher) + +================== 5.2.1 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release + +Versions of Major Components +--------------------- +Apache Tika 1.7 +Carrot2 3.9.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.2.10.v20150310 + +Detailed Change List +---------------------- + +Bug Fixes +---------------------- + +* SOLR-7588: Fix javascript bug introduced by SOLR-7409 that breaks the + dataimport screen in the admin UI. (Bill Bell via Shawn Heisey) + +* SOLR-7616: Faceting on a numeric field with a unique() subfacet function on another numeric field + can result in incorrect results or an exception. (yonik) + +* SOLR-7518: New Facet Module should respect shards.tolerant and process all non-failing shards + instead of throwing an exception. (yonik) + +* SOLR-7574: A request with a json content type but no body caused a null pointer exception (yonik) + +* SOLR-7512: SolrOutputFormat creates an invalid solr.xml in the solr home zip for MapReduceIndexerTool. + (Mark Miller, Adam McElwee) + +* SOLR-7652: Fix example/files update-script.js to work with Java 7 (ehatcher) + +* SOLR-7638: Fix new (Angular-based) admin UI Cloud pane (Upayavira via ehatcher) + +* SOLR-7655: The DefaultSolrHighlighter since 5.0 was determining if payloads were present in a way + that was slow, especially when lots of fields were highlighted. It's now fast. (David Smiley) + +* SOLR-7493: Requests aren't distributed evenly if the collection isn't present locally. + (Jeff Wartes, shalin) + +Other Changes +---------------------- + +* SOLR-7623: Fix regression from SOLR-7484 that made it impossible to override + SolrDispatchFilter#execute() and SolrDispatchFilter#sendError(). You can now override these + functions in HttpSolrCall. (ryan) + +* SOLR-7648: Expose remote IP and Host via the AuthorizationContext to be used by the authorization plugin. + (Ishan Chattopadhyaya via Anshum Gupta) + +================== 5.2.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release + +Versions of Major Components +--------------------- +Apache Tika 1.7 +Carrot2 3.9.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 9.2.10.v20150310 + +Upgrading from Solr 5.1 +----------------------- + +* A bug was introduced in Solr 4.10 that caused index time document boosts to trigger excessive field + boosts in multivalued fields -- the result being that some field norms might be excessively large. + This bug has now been fixed, but users of document boosts are strongly encouraged to re-index. + See SOLR-7335 for more details. + +* The Slice and Replica classes have been changed to use State enums instead of string constants + to track the respective stats. Advanced users with client code manipulating these objects will + need to update their code accordingly. See SOLR-7325 and SOLR-7336 for more info. + +* Solr has internally been upgraded to use Jetty 9. See SOLR-4839 for full details, but there are + a few key details all Solr users should know when upgrading: + + - It is no longer possible to run "java -jar start.jar" from inside the server directory. + The bin/solr script is the only supported way to run Solr. This is necessary to support + HTTP and HTTPS modules in Jetty which can be selectively enabled by the bin/solr scripts. + In case you have a pressing need to run solr the old way, you can run + "java -jar start.jar --module=http" to get the same behavior as before. + + - The way SSL support is configured has been changed. Before this release, + the SOLR_SSL_OPTS property configured in solr.in.sh (linux/mac) or solr.in.cmd (windows) + was used to enable/disable SSL but starting in 5.2.0, new properties named as + SOLR_SSL_KEY_STORE, SOLR_SSL_KEY_STORE_PASSWORD, SOLR_SSL_TRUST_STORE, + SOLR_SSL_TRUST_STORE_PASSWORD, SOLR_SSL_NEED_CLIENT_AUTH and SOLR_SSL_WANT_CLIENT_AUTH + have been introduced. The bin/solr scripts configure the SOLR_SSL_OPTS property + automatically based on the above new properties. + + You should *not* configure the SOLR_SSL_OPTS property directly inside solr.in.{sh,cmd}. + + - Support for SOLR_SSL_PORT property has been removed. Instead use the regular SOLR_PORT + property or specify the port while invoking the bin/solr script using the "-p" switch. + + - Furthermore, it is now possible to configure the HTTP client with + different SSL properties than the ones used for Jetty using the same files. + + - Please refer to the "Enabling SSL" section in the Solr Reference Guide for complete details. + +* Support for pathPrefix has been completely removed from Solr. Since 5.0, Solr no longer officially + supports being run as a webapp but allowed users to play around with the web.xml to have a path prefix. + That would no longer be true. See SOLR-7500 for more info. + +* The package structure under org.apache.solr.client.solrj.io has been changed to support + the Streaming Expression Language (SOLR-7377). Any code written with the 5.1 Streaming API will have to + be updated to reflect these changes. + +* Merge Policy's "noCFSRatio" is no longer set based on <useCompoundFile> element in the indexConfig section + of solrconfig.xml. This means that Solr will start using Lucene's default for MP "noCFSRatio", with this + new default Solr will decide if a segment should use cfs or not based on the size of the segment in relation + the size of the complete index. For TieredMergePolicy for example (current default), segments will use cfs + if they are less than 10% of the index, otherwise cfs is disabled. Old values for this setting + (1.0 for useCompoundFile=true and 0.0 for useCompoundFile=false) as well as any other value can be set + inside the <mergePolicy> element in solrconfig.xml. <useCompoundFile> will only apply to newly created + segments. See SOLR-7463. + + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-6637: Restore a Solr core from a backed up index. + Restore API Example - + http://localhost:8983/solr/techproducts/replication?command=restore&name=backup_name + Restore Status API Example - + http://localhost:8983/solr/techproducts/replication?command=restorestatus + (Varun Thacker, noble, shalin) + +* SOLR-7241, SOLR-7263, SOLR-7279, SOLR-7300, SOLR-7396, SOLR-7397, SOLR-7492: + Admin UI - Refactoring using AngularJS. More functionality moving the Admin + UI to Angular JS (Upayavira via Erick) + +* SOLR-7372: Limit memory consumed by LRUCache with a new 'maxRamMB' config parameter. + (yonik, shalin) + +* SOLR-7376: Return raw XML or JSON (in the appropriate writer) using DocumentTransformers. + ?fl=id,name,json_s:[json],xml_s:[xml] (ryan) + +* SOLR-7422: Optional flatter form for the JSON Facet API via a "type" parameter: + top_authors : { type:terms, field:author, limit:5 } is equivalent to + top_authors : { terms : { field:author, limit:5 } } + (yonik) + +* SOLR-7176: zkcli script can perfrom the CLUSTERPROP command without a running Solr cluster + (Hrishikesh Gadre, Per Steffensen, Noble Paul) + +* SOLR-7417: JSON Facet API - unique() is now implemented for numeric and date fields. + (yonik) + +* SOLR-7406: Add a new "facet.range.method" parameter to let users choose how to do range + faceting between an implementation based on filters (previous algorithm, using + "facet.range.method=filter") or DocValues ("facet.range.method=dv"). + Input parameters and output of both methods are the same. (Tomás Fernández Löbbe) + +* SOLR-7473: Facet Module (Json Facet API) range faceting now supports the "mincount" + parameter in range facets to supress buckets less than that count. The default + for "mincount" remains 0 for range faceting. + Example: prices:{ type:range, field:price, mincount:1, start:0, end:100, gap:10 } + (yonik) + +* SOLR-7437: Make HDFS transaction log replication factor configurable. (Mark Miller) + +* SOLR-7477: Multi-select faceting support for the Facet Module via the "excludeTags" + parameter which disregards any matching tagged filters for that facet. Example: + & q=shoes + & fq={!tag=COLOR}color:blue + & json.facet={ colors:{type:terms, field:color, excludeTags=COLOR} } + (yonik) + +* SOLR-7231: DIH-TikaEntityprocessor, create lat-lon field from Metadata + (Tim Allison via Noble Paul) + +* SOLR-6220: Rule Based Replica Assignment during collection, shard creation + and replica creation (Noble Paul) + +* SOLR-6968: New 'cardinality' option for stats.field, uses HyperLogLog to efficiently + estimate the cardinality of a field w/bounded RAM. (hossman) + +* SOLR-4392: Make it possible to specify AES encrypted password in dataconfig.xml (Noble Paul) + +* SOLR-7461: stats.field now supports individual local params for 'countDistinct' and 'distinctValues'. + 'calcdistinct' is still supported as an alias for both options (hossman) + +* SOLR-7522: Facet Module - Implement field/terms faceting over single-valued + numeric fields. (yonik) + +* SOLR-7275: Authorization framework for Solr. It defines an interface and a mechanism to create, + load, and use an Authorization plugin. (Noble Paul, Ishan Chattopadhyaya, Anshum Gupta) + +* SOLR-7377: Solr Streaming Expressions (Dennis Gove, Joel Bernstein, Steven Bower) + +* SOLR-7553: Facet Analytics Module: new "hll" function that uses HyperLogLog to calculate + distributed cardinality. The original "unique" function is still available. + Example: json.facet={ numProducts : "hll(product_id)" } + (yonik) + +* SOLR-7546: bin/post (and SimplePostTool in -Dauto=yes mode) now sends rather than skips files + without a known content type, as "application/octet-stream", provided it still is in the + allowed filetypes setting. (ehatcher) + +* SOLR-7274: Pluggable authentication module in Solr. This defines an interface and a mechanism to create, + load, and use an Authentication plugin. (Noble Paul, Ishan Chattopadhyaya, Gregory Chanan, Anshum Gupta) + +* SOLR-7379: (experimental) New spatial RptWithGeometrySpatialField, based on CompositeSpatialStrategy, + which blends RPT indexes for speed with serialized geometry for accuracy. Includes a Lucene segment based + in-memory shape cache. (David Smiley) + +* SOLR-7465, SOLR-7610: New file indexing example, under example/files. (Esther Quansah, Erik Hatcher) + +* SOLR-7468: Kerberos authenticaion plugin for Solr. This would allow running a Kerberized Solr. + (Noble Paul, Ishan Chattopadhyaya, Gregory Chanan, Anshum Gupta) + +Bug Fixes +---------------------- + +* SOLR-6709: Fix QueryResponse to deal with the "expanded" section when using the XMLResponseParser + (Varun Thacker, Joel Bernstein) + +* SOLR-7066: autoAddReplicas feature has bug when selecting replacement nodes. (Mark Miller) + +* SOLR-7370: FSHDFSUtils#recoverFileLease tries to recover the lease every one second after + the first four second wait. (Mark Miller) + +* SOLR-7369: AngularJS UI insufficient URLDecoding in cloud/tree view (janhoy) + +* SOLR-7380: SearchHandler should not try to load runtime components in inform() (Noble Paul) + +* SOLR-7385: The clusterstatus API now returns the config set used to create a collection + inside a 'configName' key. (Shai Erera, shalin) + +* SOLR-7401: Fixed a NullPointerException when concurrently creating and deleting collections, + while accessing other collections. (Shai Erera) + +* SOLR-7412: Fixed range.facet.other parameter for distributed requests. + (Will Miller, Tomás Fernández Löbbe) + +* SOLR-6087: SolrIndexSearcher makes no DelegatingCollector.finish() call when IndexSearcher + throws an expected exception. (Christine Poerschke via shalin) + +* SOLR-7420: Overseer stats are not reset on loss of ZK connection. (Jessica Cheng, shalin) + +* SOLR-7392: Fix SOLR_JAVA_MEM and SOLR_OPTS customizations in solr.in.sh being ignored + (Ramkumar Aiyengar, Ere Maijala) + +* SOLR-7426: SolrConfig#getConfigOverlay does not clean up it's resources. (Mark Miller) + +* SOLR-6665: ZkController.publishAndWaitForDownStates can return before all local cores are + marked as 'down' if multiple replicas with the same core name exist in the cluster. + (shalin) + +* SOLR-7418: Check and raise a SolrException instead of an NPE when an invalid doc id is sent + to the MLTQParser. (Anshum Gupta) + +* SOLR-7443: Implemented range faceting over date fields in the new facet module + (JSON Facet API). (yonik) + +* SOLR-7440: DebugComponent does not return the right requestPurpose for pivot facet refinements. + (shalin) + +* SOLR-7408: Listeners set by SolrCores on config directories in ZK could be removed if collections + are created/deleted in paralle against the same config set. (Shai Erera, Anshum Gupta) + +* SOLR-7450: Fix edge case which could cause `bin/solr stop` to hang forever + (Ramkumar Aiyengar) + +* SOLR-7157: initParams must support tags other than appends, defaults and, invariants (Noble Paul) + +* SOLR-7387: Facet Module - distributed search didn't work when sorting terms + facet by min, max, avg, or unique functions. (yonik) + +* SOLR-7469: Fix check-licenses to correctly detect if start.jar.sha1 is incorrect (hossman) + +* SOLR-7449: solr/server/etc/jetty-https-ssl.xml hard codes the key store file and password rather + than pulling them from the sysprops defined in solr/bin/solr.in.{sh,cmd} + +* SOLR-7470: Fix sample data to eliminate file order dependency for successful indexing, also + fixed SolrCloudExampleTest to help catch this in the future. (hossman) + +* SOLR-7478: UpdateLog#close shuts down it's executor with interrupts before running it's close logic, + possibly preventing a clean close. (Mark Miller) + +* SOLR-7494: Facet Module - unique() facet function was wildly inaccurate for high cardinality + fields. (Andy Crossen, yonik) + +* SOLR-7502: start script should not try to create configset for .system collection (Noble Paul) + +* SOLR-7514: SolrClient.getByIds fails with ClassCastException (Tom Farnworth, Ramkumar Aiyengar) + +* SOLR-7531: config API shows a few keys merged together (Noble Paul) + +* SOLR-7542: Schema API: Can't remove single dynamic copy field directive + (Steve Rowe) + +* SOLR-7472: SortingResponseWriter does not log fl parameters that don't exist. (Joel Bernstein) + +* SOLR-7545: Honour SOLR_HOST parameter with bin/solr{,.cmd} + (Ishan Chattopadhyaya via Ramkumar Aiyengar) + +* SOLR-7503: Recovery after ZK session expiration should happen in parallel for all cores + using the thread-pool managed by ZkContainer instead of a single thread. + (Jessica Cheng Mallet, Timothy Potter, shalin, Mark Miller) + +* SOLR-7335: Fix doc boosts to no longer be multiplied in each field value in multivalued fields that + are not used in copyFields (Shingo Sasaki via hossman) + +* SOLR-7585: Fix NoSuchElementException in LFUCache resulting from heavy writes + making concurrent put() calls. (Maciej Zasada via Shawn Heisey) + +* SOLR-7587: Seeding bucket versions from index when the firstSearcher event fires has a race condition + that leads to an infinite wait on VersionInfo's ReentrantReadWriteLock because the read-lock acquired + during a commit cannot be upgraded to a write-lock needed to block updates; solution is to move the + call out of the firstSearcher event path and into the SolrCore constructor. (Timothy Potter) + +* SOLR-7625: Ensure that the max value for seeding version buckets is updated after recovery even if + the UpdateLog is not replayed. (Timothy Potter) + +* SOLR-7610: Fix VelocityResponseWriter's $resource.locale to accurately report locale in use. + (ehatcher) + +* SOLR-7614: Distributed pivot facet refinement was broken due to a single correlation counter + used across multiple requests as if it was private to each request. (yonik) + + +Optimizations +---------------------- + +* SOLR-7324: IndexFetcher does not need to call isIndexStale if full copy is already needed + (Stephan Lagraulet via Varun Thacker) + +* SOLR-7547: Short circuit SolrDisptachFilter for static content request. Right now it creates + a new HttpSolrCall object and tries to process it. (Anshum Gupta) + +* SOLR-7333: Make the poll queue time a leader uses when distributing updates to replicas + configurable and use knowledge that a batch is being processed to poll efficiently. + (Timothy Potter) + +* SOLR-7332: Initialize the highest value for all version buckets with the max value from + the index or recent updates to avoid unnecessary lookups to the index to check for reordered + updates when processing new documents. (Timothy Potter, yonik) + +* SOLR-5855: DefaultSolrHighlighter now re-uses the document's term vectors instance when highlighting + more than one field. Applies to the standard and FVH highlighters. (David Smiley, Daniel Debray) + +Other Changes +---------------------- + +* SOLR-6865: Upgrade HttpClient to 4.4.1 (Shawn Heisey) + +* SOLR-7358: TestRestoreCore fails in Windows (Ishan Chattopadhyaya via Varun Thacker) + +* SOLR-7371: Make DocSet implement Accountable to estimate memory usage. (yonik, shalin) + +* SOLR-7381: Improve logging by adding node name in MDC in SolrCloud mode and adding MDC to + all thread pools. A new MDCAwareThreadPoolExecutor is introduced and usages of + Executors#newFixedThreadPool, #newSingleThreadExecutor, #newCachedThreadPool as well as + ThreadPoolExecutor directly is now forbidden in Solr. MDC keys are now exposed in thread + names automatically so that a thread dump can give hints on what the thread was doing. + Uncaught exceptions thrown by tasks in the pool are logged along with submitter's stack trace. + (shalin) + +* SOLR-7384: Fix spurious failures in FullSolrCloudDistribCmdsTest. (shalin) + +* SOLR-6692: Default highlighter changes: + - hl.maxAnalyzedChars now applies cumulatively on a multi-valied field. + - fragment ranking on a multi-valued field should be more relevant. + - hl.usePhraseHighlighter is now toggleable on a per-field basis. + - Much more extensible (get values from another source; return snippet scores and offsets). + - When using hl.maxMultiValuedToMatch with hl.preserveMulti, only count matched snippets. + (David Smiley) + +* SOLR-6886: Removed redundant size check and added missing calls to + DelegatingCollection.finish inside Grouping code. (Christine Poerschke via shalin) + +* SOLR-7421: RecoveryAfterSoftCommitTest fails frequently on Jenkins due to full index + replication taking longer than 30 seconds. (Timothy Potter, shalin) + +* SOLR-7081: Add new test case to test if create/delete/re-create collections work. + (Christine Poerschke via Ramkumar Aiyengar) + +* SOLR-7467: Upgrade t-digest to 3.1 (hossman) + +* SOLR-7471: Stop requiring docValues for interval faceting (Tomás Fernández Löbbe) + +* SOLR-7391: Use a time based expiration cache for one off HDFS FileSystem instances. + (Mark Miller) + +* SOLR-5213: Log when shard splitting unexpectedly leads to documents going to + no or multiple shards (Christine Poerschke, Ramkumar Aiyengar) + +* SOLR-7425: Improve MDC based logging format. (Mark Miller) + +* SOLR-4839: Upgrade Jetty to 9.2.10.v20150310 and restlet-jee to 2.3.0 + (Bill Bell, Timothy Potter, Uwe Schindler, Mark Miller, Steve Rowe, Steve Davids, shalin) + +* SOLR-7457: Make DirectoryFactory publishing MBeanInfo extensible. + (Mike Drob via Mark Miller) + +* SOLR-7325: Slice.getState() now returns a State enum instead of a String. This helps + clarify the states a Slice can be in, as well comparing the state of a Slice. + (Shai Erera) + +* SOLR-7336: Added Replica.getState() and removed ZkStateReader state-related constants. + You should use Replica.State to compare a replica's state. (Shai Erera) + +* SOLR-7487: Fix check-example-lucene-match-version Ant task to check luceneMatchVersion + in solr/server/solr/configsets instead of example and harden error checking / validation + logic. (hossman, Timothy Potter) + +* SOLR-7409: When there are multiple dataimport handlers defined, the admin UI + was listing them in a random order. Now they are sorted in a natural order + that handles numbers properly. (Jellyfrog via Shawn Heisey) + +* SOLR-7484: Refactor SolrDispatchFilter to extract all Solr specific implementation detail + to HttpSolrCall and also extract methods from within the current SDF.doFilter(..) logic + making things easier to manage. HttpSolrCall converts the processing to a 3-step process + i.e. Construct, Init, and Call so the context of the request would be available after Init + and before the actual call operation. (Anshum Gupta, Noble Paul) + +* SOLR-6878: Allow symmetric lists of synonyms to be added using the managed synonym REST + API to support legacy expand=true type mappings; previously the API only allowed adding + explicit mappings, with this feature you can now add a list and have the mappings + expanded when the update is applied (Timothy Potter, Vitaliy Zhovtyuk, hossman) + +* SOLR-7102: bin/solr should activate cloud mode if ZK_HOST is set (Timothy Potter) + +* SOLR-7500: Remove pathPrefix from SolrDispatchFilter as Solr no longer runs as a part + of a bigger webapp. (Anshum Gupta) + +* SOLR-7243: CloudSolrClient was always returning SERVER_ERROR for exceptions, + even when a more relevant ErrorCode was available, via SolrException. Now + the actual ErrorCode is used when available. + (Hrishikesh Gadre via Shawn Heisey) + +* SOLR-7544: CollectionsHandler refactored to be more modular (Noble Paul) + +* SOLR-7532: Removed occurrences of the unused 'commitIntervalLowerBound' property for + updateHandler elements from Solr configuration. (Marius Grama via shalin) + +* SOLR-7541: Removed CollectionsHandler#createNodeIfNotExists. All calls made to this method now call + ZkCmdExecutor#ensureExists as they were doing the same thing. Also ZkCmdExecutor#ensureExists now respects the + CreateMode passed to it. (Varun Thacker) + +* SOLR-6820: Make the number of version buckets used by the UpdateLog configurable as + increasing beyond the default 256 has been shown to help with high volume indexing + performance in SolrCloud; helps overcome a limitation where Lucene uses the request + thread to perform expensive index housekeeping work. (Mark Miller, yonik, Timothy Potter) + +* SOLR-7463: Stop forcing MergePolicy's "NoCFSRatio" based on the IWC "useCompoundFile" configuration + (Tomás Fernández Löbbe) + +* SOLR-7582: Allow auto-commit to be set with system properties in data_driven_schema_configs and + enable auto soft-commits for the bin/solr -e cloud example using the Config API. + (Timothy Potter) + +* SOLR-7183: Fix Locale blacklisting for Minikdc based tests. (Ishan Chattopadhyaya, hossman + via Anshum Gupta) + +* SOLR-7662: Refactored response writing to consolidate the logic in one place (Noble Paul) + +* SOLR-7110: Added option to optimize JavaBinCodec to minimize string Object creation (Noble Paul) + +================== 5.1.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release + +Versions of Major Components +--------------------- +Apache Tika 1.7 +Carrot2 3.9.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 8.1.10.v20130312 + +Upgrading from Solr 5.0 +----------------------- + +* SolrClient query functions now declare themselves as throwing IOException in + addition to SolrServerException, to bring them in line with the update + functions. + +* SolrRequest.process() is now final. Subclasses should instead be parameterized + by their corresponding SolrResponse type, and implement createResponse() + +* The signature of SolrDispatchFilter.createCoreContainer() has changed to take + (String,Properties) arguments + +* Deprecated the 'lib' option added to create-requesthandler as part of SOLR-6801 in 5.0 release. + Please use the add-runtimelib command + +* Tika's runtime dependency of 'jhighlight' was removed as the latter was found to + contain some LGPL-only code. Until that's resolved by Tika, you can download the + .jar yourself and place it under contrib/extraction/lib. + +* The _text catch-all field in data_driven_schema_configs has been renamed to _text_. + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-6909: Extract atomic update handling logic into AtomicUpdateDocumentMerger class + and enable subclassing. (Steve Davids, yonik) + +* SOLR-6845: Add a “buildOnStartup” option for suggesters. (Tomás Fernández Löbbe) + +* SOLR-6449: Add first class support for Real Time Get in Solrj. + (Anurag Sharma, Steve Davids via shalin) + +* SOLR-6954: SolrClient now implements Closeable, and shutdown() has been + deprecated in favour of close(). (Mark Miller, Tomás Fernández Löbbe, Alan + Woodward) + +* SOLR-4905: Allow fromIndex parameter to JoinQParserPlugin to refer to a single-sharded + collection that has a replica on all nodes where there is a replica in the to index + (Jack Lo, Timothy Potter) + +* SOLR-6648: Add support in AnalyzingInfixLookupFactory and BlendedInfixLookupFactory + for setting 'highlight' and 'allTermsRequired' in the suggester configuration. + (Boon Low, Varun Thacker via Tomás Fernández Löbbe) + +* SOLR-7083: Support managing all named components in solrconfig such as + requestHandler, queryParser, queryResponseWriter, valueSourceParser, + transformer, queryConverter (Noble Paul) + +* SOLR-7005: Spatial 2D heatmap faceting on RPT fields via new facet.heatmap with PNG and + 2D int array formats. (David Smiley) + +* SOLR-7019: Support changing field key when using interval faceting. + (Tomás Fernández Löbbe) + +* SOLR-6832: Queries be served locally rather than being forwarded to another replica. + (Sachin Goyal, Timothy Potter) + +* SOLR-1945: Add support for child docs in DocumentObjectBinder (Noble Paul, Mark Miller) + +* SOLR-7125, SOLR-7158: You can upload and download configurations via CloudSolrClient + (Alan Woodward, Ishan Chattopadhyaya) + +* SOLR-5507: Admin UI - Refactoring using AngularJS, first part (Upayavira via + Erick Erickson) + +* SOLR-7164: BBoxField defaults sub fields to not-stored (ryan) + +* SOLR-7155,SOLR-7201: All SolrClient methods now take an optional 'collection' argument + (Alan Woodward, Shawn Heisey) + +* SOLR-6359: Allow number of logs and records kept by UpdateLog to be configured + (Ramkumar Aiyengar) + +* SOLR-7189: Allow DIH to extract content from embedded documents via Tika. + (Tim Allison via shalin) + +* SOLR-6841: Visualize lucene segment information in Admin UI. + (Alexey Kozhemiakin, Michal Bienkowski, hossman, Shawn Heisey, Varun Thacker via shalin) + +* SOLR-5846: EnumField supports DocValues functionality. (Elran Dvir, shalin) + +* SOLR-4044: CloudSolrClient.connect() throws a more useful exception if the + cluster is not ready, and can now take an optional timeout argument to wait + for the cluster. (Alan Woodward, shalin, yonik, Mark Miller, Vitaliy Zhovtyuk) + +* SOLR-7073: Support adding a jar to a collections classpath (Noble Paul) + +* SOLR-7126: Secure loading of runtime external jars (Noble Paul) + +* SOLR-6349: Added support for stats.field localparams to enable/disable individual stats to + limit the amount of computation done and the amount of data returned. + eg: stats.field={!min=true max=true}field_name + (Tomas Fernandez-Lobbe, Xu Zhang, hossman) + +* SOLR-7218: lucene/solr query syntax to give any query clause a constant score. + General Form: <clause>^=<constant_score> + Example: (color:blue color:green)^=2.0 text:shoes + (yonik) + +* SOLR-7214: New Facet module with a JSON API, facet functions, aggregations, and analytics. + Any facet type can have sub facets, and facets can be sorted by arbitrary aggregation functions. + Examples: + json.facet={x:'avg(price)', y:'unique(color)'} + json.facet={count1:{query:"price:[10 TO 20]"}, count2:{query:"color:blue AND popularity:[0 TO 50]"} } + json.facet={categories:{terms:{field:cat, sort:"x desc", facet:{x:"avg(price)", y:"sum(price)"}}}} + (yonik) + +* SOLR-6141: Schema API: Remove fields, dynamic fields, field types and copy + fields; and replace fields, dynamic fields and field types. (Steve Rowe) + +* SOLR-7217: HTTP POST body is auto-detected when the client is curl and the content + type is form data (curl's default), allowing users to use curl to send + JSON or XML without having to specify the content type. (yonik) + +* SOLR-6892: Update processors can now be top-level components and they can be + specified in request to create a new custom update chain (Noble Paul) + +* SOLR-7216: Solr JSON Request API: + - HTTP search requests can have a JSON body. + - JSON request can also be passed via the "json" parameter. + - Smart merging of multiple JSON parameters: ruery parameters starting with "json." + will be merged into the JSON request. + - Legacy query parameters can also be passed in the "params" block of + the JSON request. + (yonik) + +* SOLR-7245: Temporary ZK election or connection loss should not stall indexing + due to leader initiated recovery (Ramkumar Aiyengar) + +* SOLR-6350: StatsComponent now supports Percentiles (Xu Zhang, hossman) + +* SOLR-7306: Percentiles support for the new facet module. Percentiles + can be calculated for all facet buckets and field faceting can sort + by percentile values. + Examples: + json.facet={ median_age : "percentile(age,50)" } + json.facet={ salary_percentiles : "percentile(salary,25,50,75)" } + (yonik) + +* SOLR-7307: EmbeddedSolrServer can now be started up by passing a path to a + solr home directory, or a NodeConfig object (Alan Woodward, Mike Drob) + +* SOLR-1387: Add facet.contains and facet.contains.ignoreCase options (Tom Winch + via Alan Woodward) + +* SOLR-7082: Streaming Aggregation for SolrCloud (Joel bernstein, Yonik Seeley) + +* SOLR-7212: Parameter substitution / macro expansion across entire request. + Substitution can contain further expansions and default values are supported. + Example: q=price:[ ${low:0} TO ${high} ]&low=100&high=200 + (yonik) + +* SOLR-7226: Make /query/* jmx/* , requestDispatcher/*, <listener> <initParams> + properties in solrconfig.xml editable (Noble Paul) + +* SOLR-7240: '/' redirects to '/solr/' for convenience (Martijn Koster, hossman) + +* SOLR-5911: Added payload support for term vectors. New "termPayloads" option for fields + / types in the schema, and "tv.payloads" param for the term vector component. + (Mike McCandless, David Smiley) + +* SOLR-5132: Added a new collection action MODIFYCOLLECTION (Noble Paul) + +Bug Fixes +---------------------- + +* SOLR-7046: NullPointerException when group.function uses query() function. + (Jim Musil via Erick Erickson) + +* SOLR-7072: Multiple mlt.fl does not work. (Constantin Mitocaru, shalin) + +* SOLR-6775: Creating backup snapshot results in null pointer exception. + (Ryan Hesson, Varun Thacker via shalin) + +* SOLR-5890: Delete silently fails if not sent to shard where document was + added (Ishan Chattopadhyaya, Noble Paul) + +* SOLR-7101: JmxMonitoredMap can throw an exception in clear when queryNames fails. + (Mark Miller, Wolfgang Hoschek) + +* SOLR-6214: Snapshots numberToKeep param only keeps n-1 backups. + (Mathias H., Ramana, Varun Thacker via shalin) + +* SOLR-7084: FreeTextSuggester: Better error message when doing a lookup + during dictionary build. Used to be nullpointer (janhoy) + +* SOLR-6956: OverseerCollectionProcessor and replicas on the overseer node can sometimes + operate on stale cluster state due to overseer holding the state update lock for a + long time. (Mark Miller, shalin) + +* SOLR-7104: Propagate property prefix parameters for ADDREPLICA Collections API call. + (Varun Thacker via Anshum Gupta) + +* SOLR-7113: Multiple calls to UpdateLog#init is not thread safe with respect to the + HDFS FileSystem client object usage. (Mark Miller, Vamsee Yarlagadda) + +* SOLR-7128: Two phase distributed search is fetching extra fields in GET_TOP_IDS phase. + (Pablo Queixalos, shalin) + +* SOLR-7139: Fix SolrContentHandler for TIKA to ignore multiple startDocument events. + (Chris A. Mattmann, Uwe Schindler) + +* SOLR-7178: OverseerAutoReplicaFailoverThread compares Integer objects using == + (shalin) + +* SOLR-7171: BaseDistributedSearchTestCase now clones getSolrHome() for each subclass, + and consistently uses getSolrXml(). (hossman) + +* SOLR-6657: DocumentDictionaryFactory requires weightField to be mandatory, but it shouldn't + (Erick Erickson) + +* SOLR-7206: MiniSolrCloudCluster wasn't dealing with SSL mode correctly (Alan + Woodward) + +* SOLR-4464: DIH Processed documents counter resets to zero after first entity is processed. + (Dave Cook, Shawn Heisey, Aaron Greenspan, Thomas Champagne via shalin) + +* SOLR-7209: /update/json/docs carry forward fields from previous records (Noble Paul) + +* SOLR-7195: Fixed a bug where the bin/solr shell script would incorrectly + detect another Solr process listening on the same port number. If the + requested listen port was 8983, it would match on another Solr using port + 18983 for any purpose. Also escapes the dot character in all grep commands + looking for start.jar. + (Xu Zhang via Shawn Heisey) + +* SOLR-6682: Fix response when using EnumField with StatsComponent + (Xu Zhang via hossman) + +* SOLR-7109: Indexing threads stuck during network partition can put leader into down state. + (Mark Miller, Anshum Gupta, Ramkumar Aiyengar, yonik, shalin) + +* SOLR-7092: Stop the HDFS lease recovery retries in HdfsTransactionLog on close and try + to avoid lease recovery on closed files. (Mark Miller) + +* SOLR-7285: ActionThrottle will not pause if getNanoTime first returns 0. + (Mark Miller, Gregory Chanan) + +* SOLR-7141: RecoveryStrategy: Raise time that we wait for any updates from the leader before + they saw the recovery state to have finished. (Mark Miller) + +* SOLR-7248: In legacyCloud=false mode we should check if the core was hosted on the same node before registering it + (Varun Thacker, Noble Paul, Mark Miller) + +* SOLR-7294: Migrate API fails with 'Invalid status request: notfoundretried 6times' message. + (Jessica Cheng Mallet, shalin) + +* SOLR-7254: Make an invalid negative start/rows throw a HTTP 400 error (Bad Request) instead + of causing a 500 error. (Ramkumar Aiyengar, Hrishikesh Gadre, yonik) + +* SOLR-7305: BlendedInfixLookupFactory swallows root IOException when it occurs. + (Stephan Lagraulet via shalin) + +* SOLR-7293: Fix bug that Solr server does not listen on IPv6 interfaces by default. + (Uwe Schindler, Sebastian Pesman) + +* SOLR-7298: Fix Collections API calls (SolrJ) to not add name parameter when not needed. + (Shai Erera, Anshum Gupta) + +* SOLR-7134: Replication can still cause index corruption. (Mark Miller, shalin, Mike Drob) + +* SOLR-7309: Make bin/solr, bin/post work when Solr installation directory contains spaces + (Ramkumar Aiyengar, Martijn Koster) + +* SOLR-6924: The config API forcefully refreshes all replicas in the collection to ensure all are + updated (Noble Paul) + +* SOLR-7266: The IgnoreCommitOptimizeUpdateProcessor blocks commit requests from + replicas needing to recover. (Jessica Cheng Mallet, Timothy Potter) + +* SOLR-7299: bin\solr.cmd doesn't use jetty SSL configuration. (Steve Rowe) + +* SOLR-7334: Admin UI does not show "Num Docs" and "Deleted Docs". (Erick Erickson, Timothy Potter) + +* SOLR-7338, SOLR-6583: A reloaded core will never register itself as active after a ZK session expiration + (Mark Miller, Timothy Potter) + +* SOLR-7366: Can't index example XML docs into the cloud example using bin/post due to regression in + ManagedIndexSchema's handling of ResourceLoaderAware objects used by field types (Steve Rowe, Timothy Potter) + +* SOLR-7284: HdfsUpdateLog is using hdfs FileSystem.get without turning off the cache. + (Mark Miller) + +* SOLR-7286: Using HDFS's FileSystem.newInstance does not guarantee a new instance. + (Mark Miller) + +* SOLR-7508: SolrParams.toMultiMap() does not handle arrays (Thomas Scheffler , Noble Paul) + +Optimizations +---------------------- + + * SOLR-7049: Move work done by the LIST Collections API call to the Collections + Handler (Varun Thacker via Anshum Gupta). + + * SOLR-7116: Distributed facet refinement requests would needlessly compute other types + of faceting that have already been computed. (David Smiley, Hossman) + + * SOLR-7239: improved performance of min & max in StatsComponent, as well as situations + where local params disable all stats (hossman) + + * SOLR-7050: realtime get should internally load only fields specified in fl. + (yonik, Noble Paul) + +Other Changes +---------------------- + +* SOLR-7014: Collapse identical catch branches in try-catch statements. (shalin) + +* SOLR-6500: Refactor FileFetcher in SnapPuller, add debug logging. + (Ramkumar Aiyengar via Mark Miller) + +* SOLR-7076: In DIH, TikaEntityProcessor should have support for onError=skip + (Noble Paul) + +* SOLR-7094: Better error reporting of JSON parse issues when indexing docs + (Ishan Chattopadhyaya via Timothy Potter) + +* SOLR-7103: Remove unused method params in faceting code. (shalin) + +* SOLR-6311: When performing distributed queries, SearchHandler should use path + when no qt or shard.qt parameter is specified; fix also resolves SOLR-4479. + (Steve Molloy, Timothy Potter) + +* SOLR-7112: Fix DeleteInactiveReplicaTest.deleteLiveReplicaTest test failures. + (shalin) + +* SOLR-6902: Use JUnit rules instead of inheritance with distributed Solr + tests to allow for multiple tests without the same class. + (Ramkumar Aiyengar, Erick Erickson, Mike McCandless) + +* SOLR-7032: Clean up test remnants of old-style solr.xml (Erick Erickson) + +* SOLR-7145: SolrRequest is now parametrized by its response type (Alan + Woodward) + +* SOLR-7142: Fix TestFaceting.testFacets. (Michal Kroliczek via shalin) + +* SOLR-7156: Fix test failures due to resource leaks on windows. + (Ishan Chattopadhyaya via shalin) + +* SOLR-7147: Introduce new TrackingShardHandlerFactory for monitoring what requests + are sent to shards during tests. (hossman, shalin) + +* SOLR-7160: Rename ConfigSolr to NodeConfig, and decouple it from xml + representation (Alan Woodward) + +* SOLR-7166: Encapsulate JettySolrRunner configuration (Alan Woodward) + +* SOLR-7130: Make stale state notification work without failing the requests + (Noble Paul, shalin) + +* SOLR-7151: SolrClient query methods throw IOException (Alan Woodward) + +* SOLR-7179: JettySolrRunner no longer passes configuration to + SolrDispatchFilter via system properties, but instead uses a Properties + object in the servlet context (Alan Woodward) + +* SOLR-6275: Improve accuracy of QTime reporting (Ramkumar Aiyengar) + +* SOLR-7174: DIH should reset TikaEntityProcessor so that it is capable + of re-use (Alexandre Rafalovitch , Gary Taylor via Noble Paul) + +* SOLR-6804: Untangle SnapPuller and ReplicationHandler (Ramkumar Aiyengar) + +* SOLR-7180: MiniSolrCloudCluster will startup and shutdown its jetties in + parallel (Alan Woodward, Tomás Fernández Löbbe, Vamsee Yarlagadda) + +* SOLR-7173: Fix ReplicationFactorTest on Windows by adding better retry + support after seeing no response exceptions. (Ishan Chattopadhyaya via Timothy Potter) + +* SOLR-7246: Speed up BasicZkTest, TestManagedResourceStorage (Ramkumar Aiyengar) + +* SOLR-7258: Forbid MessageFormat.format and MessageFormat single-arg constructor. + (shalin) + +* SOLR-7162: Remove unused SolrSortField interface. (yonik, Connor Warrington via shalin) + +* SOLR-6414: Update to Hadoop 2.6.0. (Mark Miller) + +* SOLR-6673: MDC based logging of collection, shard, replica, core + (Ishan Chattopadhyaya , Noble Paul) + +* SOLR-7291: Test indexing on ZK disconnect with ChaosMonkey tests (Ramkumar Aiyengar) + +* SOLR-7203: Remove buggy no-op retry code in HttpSolrClient (Alan Woodward, + Mark Miller, Greg Solovyev) + +* SOLR-7202: Remove deprecated string action types in Overseer and OverseerCollectionProcessor - + "deletecollection", "createcollection", "reloadcollection", "removecollection", "removeshard". + (Varun Thacker, shalin) + +* SOLR-7290: Rename catchall _text field in data_driven_schema_configs + to _text_ (Steve Rowe) + +* SOLR-7346: Stored XSS in Admin UI Schema-Browser page and Analysis page (Mei Wang via Timothy Potter) + +================== 5.0.0 ================== + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release. + +NOTE: Solr 5.0 only supports creating and removing SolrCloud collections through + the collections API, unlike previous versions. While not using the + collections API may still work in 5.0, it is unsupported, not recommended, + and the behavior will change in a 5.x release. + +Versions of Major Components +--------------------- +Apache Tika 1.7 +Carrot2 3.9.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 +Jetty 8.1.10.v20130312 + +Upgrading from Solr 4.x +---------------------- + +* Apache Solr has no support for Lucene/Solr 3.x and earlier indexes anymore. + Be sure to run Lucene's IndexUpgrader on the previous 4.10 version if you might + still have old segments in your index. Alternatively fully optimize your index + with Solr 4.10 to make sure it consists only of one up-to-date index segment. + +* The "file" attribute of infoStream in solrconfig.xml is removed. Control this + via your logging configuration (org.apache.solr.update.LoggingInfoStream) instead. + +* UniqFieldsUpdateProcessorFactory no longer supports the <lst named="fields"> init + param style that was deprecated in Solr 4.5. If you are still using this syntax, + update your configs to use <arr name="fieldName"> instead. See SOLR-4249 for more + details. + +* The following legacy numeric and date field types, deprecated in Solr 4.8, are no + longer supported: BCDIntField, BCDLongField, BCDStrField, IntField, LongField, + FloatField, DoubleField, SortableIntField, SortableLongField, SortableFloatField, + SortableDoubleField, and DateField. Convert these types in your schema to the + corresponding Trie-based field type and then re-index. See SOLR-5936 for more + information. + +* getAnalyzer() in IndexSchema and FieldType that was deprecated in Solr 4.9 has + been removed. Use getIndexAnalyzer() instead. See SOLR-6022 for more information. + +* The spellcheck response format has changed, affecting xml and json clients. In + particular, the "correctlySpelled" and "collations" subsections have been moved outside + the "suggestions" subsection, and now are directly under "spellcheck". + See SOLR-3029 for more information. + +* The CollectionsAPI SolrJ calls createCollection(), reloadCollection(), + deleteCollection(), requestStatus(), createShard(), splitShard(), deleteShard(), + createAlias() and deleteAlias() which were deprecated in 4.11 have been removed. + The new usage involves a builder style construction of the call. + +* The OVERSEERSTATUS API returns new key names for operations such as "create" + for "createcollection", "delete" for "removecollection" and "deleteshard" for + "removeshard". + +* If you have been using the /update/json/docs to index documents, SOLR-6617 introduces + backward incompatible change. the key names created are fully qualified paths of keys . + If you need the old functionality back , please add an extra parameter f=/** + example: /update/json/docs?f=/** + +* Bugs fixed in several ValueSource functions may result in different behavior in + situations where some documents do not have values for fields wrapped in other value + sources. Users who want to preserve the previous behavior may need to wrap fields + in the "def()" function. Example: changing "fl=sum(fieldA,fieldB)" to + "fl=sum(def(fieldA,0.0),def(fieldB,0.0))". See LUCENE-5961 for more details. + +* AdminHandlers is deprecated, /admin/* are implicitly defined, /get, /replication and + handlers are also implicitly registered (refer to SOLR-6792) + +* SolrCore.reload(ConfigSet coreConfig, SolrCore prev) was deprecated in 4.10.3 and + removed in 5.0. use SolrCore.reload(ConfigSet coreConfig). See SOLR-5864. + +* The "termIndexInterval" option in solrconfig.xml has been a No-Op in the default codec + since Solr 4.0, and has been removed completely in 5.0. If you get an "Illegal parameter + 'termIndexInterval'" error when upgrading, you can safely remove this option from your + configs. If you have a strong need to configure this, you must explicitly configure your + schema with a custom codec. See SOLR-6560 and for more details. + +* The "checkIntegrityAtMerge" option in solrconfig.xml is now a No-Op and should be removed + from any solrconfig.xml files -- these integrity checks are now done automatically at a very + low level during the segment merging process. See SOLR-6834 for more details. + +* SimplePostTool (post.jar) no longer defaults to collection1, making either of core/collection + name or update URL mandatory. An existing call without an explicit update URL needs to now + have the core/collection name passed as "-Dc=<collection/core name>" e.g.: + java -jar -Dc=<collection_name> post.jar *.xml (new call with collection name) + See SOLR-6852 for more details. + +* Relative paths specified in the solr.xml coreRootDirectory parameter for core + discovery are now resolved relative to SOLR_HOME, rather than cwd. See + SOLR-6718. + +* SolrServer and associated classes have been deprecated. Applications using + SolrJ should use the equivalent SolrClient classes instead. + +* Spatial fields originating from Solr 4 (e.g. SpatialRecursivePrefixTreeFieldType, BBoxField) + have the 'units' attribute deprecated, now replaced with 'distanceUnits'. If you change it to + a unit other than 'degrees' (or if you don't specify it, which will default to kilometers if + geo=true), then be sure to update maxDistErr as it's in those units. If you keep units=degrees + then it should be backwards compatible but you'll get a deprecation warning on startup. See + SOLR-6797. + +* The <nrtMode> configuration in solrconfig.xml has been discontinued and should be removed from + solrconfig.xml. Solr defaults to using NRT searchers regardless of the value in configuration + and a warning is logged on startup if the solrconfig.xml has <nrtMode> specified. + +* There was an old spatial syntax to specify a circle using Circle(x,y d=...) which should be + replaced with simply using {!geofilt} (if you can) or BUFFER(POINT(x y),d). Likewise a rect syntax + comprised of minX minY maxX maxY that should now be replaced with + ENVELOPE(minX, maxX, maxY, minY). + +* Due to changes in the underlying commons-codec package, users of the BeiderMorseFilterFactory + will need to rebuild their indexes after upgrading. See LUCENE-6058 for more details. + +* CachedSqlEntityProcessor has been removed, use SqlEntityProcessor with the + cacheImpl parameter. + +* HttpDataSource has been removed, use URLDataSource instead. + +* LegacyHTMLStripCharFilter has been removed + +* CoreAdminRequest.persist() call has been removed. All changes made via + CoreAdmin are persistent. + +* SpellCheckResponse.getSuggestions() and getSuggestionFrequencies() have been + removed, use getAlternatives() and getAlternativeFrequencies() instead. + +* SolrQuery deprecated methods have been removed: + - setMissing() is now setFacetMissing() + - getFacetSort() is now getFacetSortString() + - setFacetSort(boolean) should instead use setFacetSort(String) with + FacetParams.FACET_SORT_COUNT or FacetParams.FACET_SORT_INDEX + - setSortField(String, ORDER) should use setSort(SortClause) + - addSortField(String, ORDER) should use addSort(SortClause) + - removeSortField(String, ORDER) should use removeSort(SortClause) + - getSortFields() should use getSorts() + - set/getQueryType() should use set/getRequestHandler() + +* ClientUtil deprecated date methods have been removed, use DateUtil instead + +* FacetParams.FacetDateOther has been removed, use FacetRangeOther + +* ShardParams.SHARD_KEYS has been removed, use ShardParams._ROUTE_ + +* The 'old-style' solr.xml format is no longer supported, and cores must be + defined using core.properties files. See + https://cwiki.apache.org/confluence/display/solr/Format+of+solr.xml + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-6103: Added DateRangeField for indexing date ranges, especially multi-valued ones. + Supports facet.range, DateMath, and is mostly interoperable with TrieDateField. + Based on LUCENE-5648. (David Smiley) + +* SOLR-6403: TransactionLog replay status logging. (Mark Miller) + +* SOLR-4580: Support for protecting content in ZooKeeper. (Per Steffensen, Mark Miller) + +* SOLR-6365: specify appends, defaults, invariants outside of the request handler. + (Noble Paul, Erik Hatcher, shalin) + +* SOLR-5097: Schema API: Add REST support for adding dynamic fields to the schema. + (Steve Rowe) + +* SOLR-5098: Schema API: Add REST support for adding field types to the schema. + (Timothy Potter) + +* SOLR-5473 : Split clusterstate.json per collection and watch states selectively + (Noble Paul, Mark Miller, shalin, Jessica Cheng Mallet, Timothy Potter, Anshum Gupta) + +* SOLR-5474 : Support for SOLR-5473 in SolrJ (Timothy Potter, Noble Paul, Mark Miller) + +* SOLR-5810 : Support for SOLR-5473 in solr admin UI (Timothy Potter, Noble Paul) + +* SOLR-6482: Add an onlyIfDown flag for DELETEREPLICA collections API command + (Erick Erickson) + +* SOLR-6354: stats.field can now be used to generate stats over the numeric results + of arbitrary functions, ie: stats.field={!func}product(price,popularity) + (hossman) + +* SOLR-6485: ReplicationHandler should have an option to throttle the speed of + replication (Varun Thacker, Noble Paul) + +* SOLR-6543: Give HttpSolrClient the ability to send PUT requests (Gregory Chanan) + +* SOLR-5986: Don't allow runaway queries from harming Solr cluster health or search + performance (Anshum Gupta, Steve Rowe, Robert Muir) + +* SOLR-6565: SolrRequest support for query params (Gregory Chanan) + +* SOLR-6476: Create a bulk mode for schema API (Noble Paul, Steve Rowe) + +* SOLR-6512: Add a collections API call to add/delete arbitrary properties + to a specific replica. Optionally adding sliceUnique=true will remove + this property from all other replicas within a particular slice. + (Erick Erickson) + +* SOLR-6513: Add a collectionsAPI call BALANCESLICEUNIQUE. Allows the even + distribution of custom replica properties across nodes making up a + collection, at most one node per slice will have the property. + +* SOLR-6605: Make ShardHandlerFactory maxConnections configurable. + (Christine Poerschke via shalin) + +* SOLR-6585: RequestHandlers can optionally handle sub paths as well (Noble Paul) + +* SOLR-6617: /update/json/docs path will use fully qualified node names by default + (Noble Paul) + +* SOLR-4715: Add CloudSolrClient constructors which accept a HttpClient instance. + (Hardik Upadhyay, Shawn Heisey, shalin) + +* SOLR-5992: add "removeregex" as an atomic update operation + (Vitaliy Zhovtyuk via Erick Erickson) + +* SOLR-6633: /update/json/docs path can now save the underlying json doc asa string field + and better support added to the default example (Noble Paul) + +* SOLR-6650: Add optional slow request logging at WARN level + (Jessica Cheng Mallet via Timothy Potter) + +* SOLR-6655: SimplePostTool now features -Dhost, -Dport, and -Dc (for core/collection) + properties to allow easier overriding of just the right piece of the Solr URL. + (ehatcher) + +* SOLR-6248: MoreLikeThis QParser that accepts a document id and returns documents that + have similar content. It works in standalone/cloud mode and shares logic with the + Lucene MoreLikeThis class (Anshum Gupta). + +* SOLR-6670: change BALANCESLICEUNIQUE to BALANCESHARDUNIQUE. Also, the parameter + for ADDREPLICAPROP that used to be sliceUnique is now shardUnique. (Erick Erickson) + +* SOLR-6351: Stats can now be nested under pivot values by adding a 'stats' local param to + facet.pivot which refers to a 'tag' local param in one or more stats.field params. + (hossman, Vitaliy Zhovtyuk, Steve Molloy) + +* SOLR-6533: Support editing common solrconfig.xml values (Noble Paul) + +* SOLR-6607: Managing requesthandlers through API (Noble Paul) + +* SOLR-4799: faster join using join="zipper" aka merge join for nested DIH EntityProcessors + (Mikhail Khludnev via Noble Paul) + +* SOLR-6787: API to manage blobs in Solr (Noble Paul) + +* SOLR-6801: Load RequestHandler from blob store (Noble Paul) + +* SOLR-1632: Support Distributed IDF (Andrzej Bialecki, Mark Miller, Yonik Seeley, + Robert Muir, Markus Jelsma, Vitaliy Zhovtyuk, Anshum Gupta) + +* SOLR-6729: createNodeSet.shuffle=(true|false) support for /admin/collections?action=CREATE. + (Christine Poerschke, Ramkumar Aiyengar via Mark Miller) + +* SOLR-6851: Scripts to support installing and running Solr as a service on Linux + (Timothy Potter, Hossman, Steve Rowe) + +* SOLR-6770: Add/edit param sets and use them in Requests (Noble Paul) + +* SOLR-6879: Have an option to disable autoAddReplicas temporarily for all collections. + (Varun Thacker via Steve Rowe) + +* SOLR-6435: Add bin/post script to simplify posting content to Solr (Erik Hatcher) + +* SOLR-6761: Ability to ignore commit and/or optimize requests from clients when running in + SolrCloud mode using the IgnoreCommitOptimizeUpdateProcessorFactory. (Timothy Potter) + +* SOLR-6797: Spatial fields that used to require units=degrees like + SpatialRecursivePrefixTreeFieldType (RPT) now take distanceUnits=degrees|kilometers|miles + instead. It is applied to nearly all distance measurements involving the field: maxDistErr, + distErr, d, geodist, score=distance|area|area2d. score now accepts these units as well. It does + NOT affect distances embedded in WKT strings like BUFFER(POINT(200 10),0.2)). + (Ishan Chattopadhyaya, David Smiley) + +* SOLR-6766: Expose HdfsDirectoryFactory Block Cache statistics via JMX. + (Mike Drob, Mark Miller) + +* SOLR-2035: Add a VelocityResponseWriter $resource tool for locale-specific string lookups. + (Erik Hatcher) + +* SOLR-6916: Toggle payload support for the default highlighter via hl.payloads. It's auto + enabled when the index has payloads. (David Smiley) + +* SOLR-6581: Efficient DocValues support and numeric collapse field implementations + for Collapse and Expand (Joel Bernstein) + +* SOLR-6937: In schemaless mode ,replace spaces and special characters with underscore (Noble Paul) + +* SOLR-5147: Support child documents in DIH + (Vadim Kirilchuk, Shawn Heisey, Thomas Champagne, Mikhail Khludnev via Noble Paul) + +Bug Fixes +---------------------- + +* SOLR-4895: An error should be returned when a rollback is attempted in SolrCloud mode. + (Vamsee Yarlagadda via Mark Miller) + +* SOLR-6424: The hdfs block cache BLOCKCACHE_WRITE_ENABLED is not defaulting to false like it + should. (Mark Miller) + +* SOLR-6426: SolrZkClient clean can fail due to a race with children nodes. (Mark Miller) + +* SOLR-5966: Admin UI Menu is fixed and doesn't respect smaller viewports. + (Aman Tandon, steffkes via shalin) + +* SOLR-4406: Fix RawResponseWriter to respect 'base' writer + (Steve Davids, hossman) + +* SOLR-6297: Fix WordBreakSolrSpellChecker to not lose suggestions in shard/cloud + environments (James Dyer) + +* SOLR-6467: bin/solr script should direct stdout/stderr when starting in the background + to the solr-PORT-console.log in the logs directory instead of bin. (Timothy Potter) + +* SOLR-6187: SOLR-6154: facet.mincount ignored in range faceting using distributed search + NOTE: This does NOT fixed for the (deprecated) facet.date idiom, use facet.range + instead. (Erick Erickson, Zaccheo Bagnati, Ronald Matamoros, Vamsee Yalargadda) + +* SOLR-6457: LBHttpSolrClient: ArrayIndexOutOfBoundsException risk if counter overflows + (longkey via Noble Paul) + +* SOLR-6499: Log warning about multiple update request handlers + (Noble Paul, Andreas Hubold, hossman) + +* SOLR-6507: Fixed several bugs involving stats.field used with local params (hossman) + +* SOLR-6481: CLUSTERSTATUS should check if the node hosting a replica is live when + reporting replica status (Timothy Potter) + +* SOLR-6484: SolrCLI's healthcheck action needs to check live nodes as part of reporting + the status of a replica (Timothy Potter) + +* SOLR-6540 Fix NPE from strdist() func when doc value source does not exist in a doc (hossman) + +* SOLR-6624 Spelling mistakes in the Java source (Hrishikesh Gadre) + +* SOLR-6307: Atomic update remove does not work for int array or date array + (Anurag Sharma , noble) + +* SOLR-6224: Post soft-commit callbacks are called before soft commit actually happens. + (shalin) + +* SOLR-6591: Overseer can use stale cluster state and lose updates for collections + with stateFormat > 1. (shalin) + +* SOLR-6631: DistributedQueue spinning on calling zookeeper getChildren() + (Jessica Cheng Mallet, Mark Miller, Timothy Potter) + +* SOLR-6579: SnapPuller Replication blocks clean shutdown of tomcat + (Philip Black-Knight via Noble Paul) + +* SOLR-6721: ZkController.ensureReplicaInLeaderInitiatedRecovery puts replica + in local map before writing to ZK. (shalin) + +* SOLR-6679: Disabled suggester component from techproduct solrconfig.xml since + it caused long startup times on large indexes even when it wasn't used. + (yonik, hossman) + +* SOLR-6738: Admin UI - Escape Data on Plugins-View (steffkes) + +* SOLR-3774: Solr adds RequestHandler SolrInfoMBeans twice to the JMX server. + (Tomás Fernández Löbbe, hossman, Mark Miller) + +* SOLR-6763: Shard leader elections should not persist across session expiry + (Alan Woodward, Mark Miller) + +* SOLR-3881: Avoid OOMs in LanguageIdentifierUpdateProcessor: + - Added langid.maxFieldValueChars and langid.maxTotalChars params to limit + input, by default 10k and 20k chars, respectively. + - Moved input concatenation to Tika implementation; the langdetect + implementation instead appends each input piece via the langdetect API. + (Vitaliy Zhovtyuk, Tomás Fernández Löbbe, Rob Tulloh, Steve Rowe) + +* SOLR-6626: NPE in FieldMutatingUpdateProcessor when indexing a doc with + null field value (Noble Paul) + +* SOLR-6604: SOLR-6812: Fix NPE with distrib.singlePass=true and expand + component. Increased test coverage of expand component with docValues. + (Christine Poerschke, Per Steffensen, shalin) + +* SOLR-6718: Core discovery was walking paths relative to the Jetty working + directory, rather than SOLR_HOME. (Andreas Hubold, Alan Woodward) + +* SOLR-6864: Support registering searcher listeners in SolrCoreAware.inform(SolrCore) + method. Existing components rely on this. (Tomás Fernández Löbbe) + +* SOLR-6850: AutoAddReplicas makes a call to wait to see live replicas that times + out after 30 milliseconds instead of 30 seconds. (Varun Thacker via Mark Miller) + +* SOLR-6397: zkcli script put/putfile should allow overwriting an existing znode's data + (Timothy Potter) + +* SOLR-6873: Lib relative path is incorrect for techproduct configset + (Alexandre Rafalovitch via Erick Erickson) + +* SOLR-6899: Change public setter for CollectionAdminRequest.action to protected. + (Anshum Gupta) + +* SOLR-6779: fix /browse for schemaless example (ehatcher) + +* SOLR-6874: There is a race around SocketProxy binding to it's port the way we setup + JettySolrRunner and SocketProxy. (Mark Miller, Timothy Potter) + +* SOLR-6735: Make CloneFieldUpdateProcessorFactory null safe (Steve Davids via ehatcher) + +* SOLR-6907: URLEncode documents directory in MorphlineMapperTest to handle spaces etc. + in file name. (Ramkumar Aiyengar via Erick Erickson) + +* SOLR-6880: Harden ZkStateReader to expect that getCollectionLive may return null + as it's contract states. (Mark Miller, shalin) + +* SOLR-6643: Fix error reporting & logging of low level JVM Errors that occur when + loading/reloading a SolrCore (hossman) + +* SOLR-6839: Direct routing with CloudSolrServer will ignore the Overwrite document option. + (Mark Miller) + +* SOLR-6793: ReplicationHandler does not destroy all of it's created SnapPullers. + (Mark Miller) + +* SOLR-6946: Document -p port option for the create_core and create_collection actions in + bin/solr (Timothy Potter) + +* SOLR-6923: AutoAddReplicas also consults live_nodes to see if a state change has happened. + (Varun Thacker via Anshum Gupta) + +* SOLR-6941: DistributedQueue#containsTaskWithRequestId can fail with NPE. (Mark Miller) + +* SOLR-6764: Field types need to be re-informed after reloading a managed schema from ZK + (Timothy Potter) + +* SOLR-6931: We should do a limited retry when using HttpClient. + (Mark Miller, Hrishikesh Gadre, Gregory Chanan) + +* SOLR-7004: Add a missing constructor for CollectionAdminRequest.BalanceShardUnique that + sets the collection action. (Anshum Gupta) + +* SOLR-6993: install_solr_service.sh won't install on RHEL / CentOS + (David Anderson via Timothy Potter) + +* SOLR-6928: solr.cmd stop works only in english (john.work, Jan Høydahl, Timothy Potter) + +* SOLR-7011: Delete collection returns before collection is actually removed. + (Christine Poerschke via shalin) + +* SOLR-6640: Close searchers before rollback and recovery to avoid index corruption. + (Robert Muir, Varun Thacker, shalin) + +* SOLR-6847: LeaderInitiatedRecoveryThread compares wrong replica's state with lirState. + (shalin) + +* SOLR-6856: Restore ExtractingRequestHandler's ability to capture all HTML tags when + parsing (X)HTML. (hossman, Uwe Schindler, ehatcher, Steve Rowe) + +* SOLR-7024: Improved error messages when java is not found by the bin/solr + shell script, particularly when JAVA_HOME has an invalid location. + (Shawn Heisey) + +* SOLR-7038: Validate the presence of configset before trying to create a collection. + (Anshum Gupta, Mark Miller) + +* SOLR-7037: bin/solr start -e techproducts -c fails to start Solr in cloud mode + (Timothy Potter) + +* SOLR-7016: Fix bin\solr.cmd to work in a directory with spaces in the name. + (Timothy Potter, Uwe Schindler) + +* SOLR-6969: When opening an HDFSTransactionLog for append we must first attempt to recover + it's lease to prevent data loss. (Mark Miller, Praneeth Varma, Colin McCabe) + +* SOLR-7067: bin/solr won't run under bash 4.2+. (Steve Rowe) + +* SOLR-7068: Collapse on numeric field breaks when min/max values are negative. + (Joel Bernstein) + +* SOLR-6780: Fixed a bug in how default/appends/invariants params were affecting the set + of all "keys" found in the request parameters, resulting in some key=value param pairs + being duplicated. This was noticeably affecting some areas of the code where iteration + was done over the set of all params: + - literal.* in ExtractingRequestHandler + - facet.* in FacetComponent + - spellcheck.[dictionary name].* and spellcheck.collateParam.* in SpellCheckComponent + - olap.* in AnalyticsComponent + (Alexandre Rafalovitch & hossman) + +* SOLR-6920: A replicated index can end up corrupted when small files end up with the same + file name and size. (Varun Thacker, Mark Miller) + +* SOLR-7033, SOLR-5961: RecoveryStrategy should not publish any state when + closed / cancelled and there should always be a pause between recoveries + even when recoveries are rapidly stopped and started as well as when a + node attempts to become the leader for a shard. + (Mark Miller, Maxim Novikov) + +* SOLR-6693: bin\solr.cmd doesn't support 32-bit JRE/JDK running on Windows due to + parenthesis in JAVA_HOME. (Timothy Potter, Christopher Hewitt, Jan Høydahl) + +* SOLR-12662: Reproducing TestPolicy failures: NPE and NoClassDefFoundError. + (Steve Rowe) + +Optimizations +---------------------- + +* SOLR-6603: LBHttpSolrClient - lazily allocate skipped-zombie-servers list. + (Christine Poerschke via shalin) + +* SOLR-6554: Speed up overseer operations avoiding cluster state reads from + zookeeper at the start of each loop and instead relying on local state and + compare-and-set writes. This change also adds batching for consecutive messages + belonging to the same collection with stateFormat=2. (shalin) + +* SOLR-6680: DefaultSolrHighlighter can sometimes avoid CachingTokenFilter with + hl.usePhraseHighlighter, and can be more efficient handling data from term vectors. + (David Smiley) + +* SOLR-6666: Dynamic copy fields are considering all dynamic fields, causing + a significant performance impact on indexing documents. (Liram Vardi via Erick + Erickson, Steve Rowe) + +Other Changes +---------------------- + +* SOLR-4622: Hardcoded SolrCloud defaults for hostContext and hostPort that + were deprecated in 4.3 have been removed completely. (hossman) + +* SOLR-5936: Removed deprecated non-Trie-based numeric & date field types. + (Steve Rowe) + +* SOLR-6169: Finish removal of CoreAdminHandler handleAlias action begun in 4.9 + (Alan Woodward) + +* SOLR-6215: TrieDateField should directly extend TrieField instead of + forwarding to a wrapped TrieField. (Steve Rowe) + +* SOLR-3029: Changes to spellcheck response format (Nalini Kartha via James Dyer) + +* SOLR-3957: Removed RequestHandlerUtils#addExperimentalFormatWarning(), which + removes "experimental" warning from two places: replication handler details + command and DataImportHandler responses. (ehatcher) + +* SOLR-6073: Remove helper methods from CollectionsRequest (SolrJ) for CollectionsAPI + calls and move to a builder design for the same. (Varun Thacker, Anshum Gupta) + +* SOLR-6519: Make DirectoryFactory#create() take LockFactory. + (Uwe Schindler) + +* SOLR-6400: SolrCloud tests are not properly testing session expiration. (Mark Miller) + +* LUCENE-5650: Tests can no longer write to CWD. Update log dir is now made relative + to the instance dir if it is not an absolute path. (Ryan Ernst, Dawid Weiss) + +* SOLR-6390: Remove unnecessary checked exception for CloudSolrClient + constructors, improve javadocs for CloudSolrClient constructors. + (Steve Davids via Shawn Heisey) + +* LUCENE-5901: Replaced all occurrences of LUCENE_CURRENT with LATEST for luceneMatchVersion. + (Ryan Ernst) + +* SOLR-6445: Upgrade Noggit to version 0.6 to support more flexible JSON input (Noble Paul, Yonik Seeley) + +* SOLR-6073: Remove helper methods from CollectionsRequest (SolrJ) for CollectionsAPI + calls and move to a builder design for the same. (Varun Thacker, Anshum Gupta) + +* SOLR-5322: core discovery can fail w/NPE and no explanation if a non-readable directory exists + (Said Chavkin, Erick Erickson) + +* SOLR-6488, SOLR-6991: Update to Apache Tika 1.7. This adds support for parsing Outlook PST and + Matlab MAT files. Parsing for NetCDF files was removed because of license issues; if you need + support for this format, download the parser JAR yourself and add it to contrib/extraction/lib + folder: http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/ + (Uwe Schindler) + +* SOLR-6115: Cleanup enum/string action types in Overseer, OverseerCollectionProcessor and + CollectionHandler. (Erick Erickson, shalin) + +* SOLR-6453: Stop throwing an error message from Overseer when node exits (Ramkumar Aiyengar, Noble Paul) + +* SOLR-6550: Provide simple mechanism for passing additional metadata / context about a server-side + SolrException back to the client-side (Timothy Potter) + +* SOLR-6249: Schema API changes return success before all cores are updated; client application + can provide the optional updateTimeoutSecs parameter to cause the server handling the + managed schema update to block until all replicas of the same collection have processed the + update or until the specified timeout is reached (Timothy Potter) + +* SOLR-6597: SolrIndexConfig parameter in one of the SolrIndexSearcher constructor has been removed. + It was just passed and never used via that constructor. (Anshum Gupta) + +* SOLR-5852: Add CloudSolrClient helper method to connect to a ZK ensemble. (Varun Thacker, Furkan KAMACI, + Shawn Heisey, Mark Miller, Erick Erickson via shalin) + +* SOLR-6592: Avoid waiting for the leader to see the down state if that leader is not live. + (Timothy Potter) + +* SOLR-6641: SystemInfoHandler should include the zkHost the node is using (when running in solrcloud mode) + (Timothy Potter) + +* SOLR-6295: Fix child filter query creation to never match parent docs in SolrExampleTests. + (Varun Thacker, Mikhail Khludnev via shalin) + +* SOLR-6578: Update commons-io dependency to the latest 2.4 version + (Steve Rowe, Shawn Heisey) + +* SOLR-6651: Fix wrong timeout logged in waitForReplicasToComeUp. (shalin) + +* SOLR-6698: Solr is not consistent wrt ZkCredentialsProvider / ZkCredentialProvider. + References to zkCredentialProvider in System properties or configurations should be + changed to zkCredentialsProvider. (Gregory Chanan) + +* SOLR-6715: ZkSolrResourceLoader constructors accept a parameter called 'collection' + but it should be 'configName'. (shalin) + +* SOLR-6697: bin/solr start scripts allow setting SOLR_OPTS in solr.in.* (janhoy) + +* SOLR-6739: Admin UI - Sort list of command line args (steffkes) + +* SOLR-6740: Admin UI - improve Files View (steffkes) + +* SOLR-6570: Run SolrZkClient session watch asynchronously. + (Ramkumar Aiyengar via Mark Miller) + +* SOLR-6747: Add an optional caching option as a workaround for SOLR-6586. + (Mark Miller, Gregory Chanan) + +* SOLR-6459: Normalize logging of operations in Overseer and log current queue size. + (Ramkumar Aiyengar, shalin via Mark Miller) + +* SOLR-6754: ZkController.publish doesn't use the updateLastState parameter. + (shalin) + +* SOLR-6751: Exceptions thrown in the analysis chain in DirectUpdateHandler2 + should return a BAD_REQUEST status (Alan Woodward) + +* SOLR-6792: deprecate AdminHandlers, Clean up solrconfig.xml of + unnecessary plugin definitions, implicit registration of /replication, + /get and /admin/* handlers (Noble Paul) + +* SOLR-5864: Remove previous SolrCore as parameter on reload. + (Tomás Fernández Löbbe) + +* SOLR-4792: Stop shipping a .war. (Robert Muir, Ramkumar Aiyengar, Mark Miller) + +* SOLR-6799: Update Saxon-HE to 9.6.0-2. (Mark Miller) + +* SOLR-6454: Suppress EOFExceptions in SolrDispatchFilter. + (Ramkumar Aiyengar via Mark Miller) + +* SOLR-6370: Allow tests to report/fail on many ZK watches being parallelly + requested on the same data (Ramkumar Aiyengar via Timothy Potter) + +* SOLR-6752: Buffer Cache allocate/lost metrics should be exposed. + (Mike Drob via Mark Miller) + +* SOLR-6560: Purge termIndexInterval from example/test configs + (Tom Burton-West, hossman) + +* SOLR-6773: Remove the multicore example as the DIH and cloud examples + illustrate multicore behavior (hossman, Timothy Potter) + +* SOLR-6834: Warn if checkIntegrityAtMerge is configured. This option is no longer meaningful + since the checks are done automatically at a very low level in the segment merging. + This warning will become an error in Solr 6.0. (hossman) + +* SOLR-6833: Examples started with bin/solr -e should use a solr.solr.home directory under + the example directory instead of server/solr. (Alexandre Rafalovitch, Anshum Gupta, hossman, + Timothy Potter) + +* SOLR-6826: fieldType capitalization is not consistent with the rest of case-sensitive field names. + (Alexandre Rafalovitch via Erick Erickson) + +* SOLR-6849: HttpSolrClient.RemoteSolrException reports the URL of the remote + host where the exception occurred. (Alan Woodward) + +* SOLR-6852: SimplePostTool no longer defaults to collection1 making core/collection/update URL + mandatory. (Anshum Gupta) + +* SOLR-6861: post.sh from exampledocs directory has been removed as there no longer is a default update + URL. (Anshum Gupta) + +* SOLR-5922: Add support for adding core properties to SolrJ Collection Admin Request calls. + (Varun Thacker via Anshum Gupta). + +* SOLR-6523: Provide SolrJ support for specifying stateFormat while creating Collections. + (Anshum Gupta) + +* SOLR-6881: Add split.key support for SPLITSHARD via SolrJ (Anshum Gupta) + +* SOLR-6883: CLUSTERPROP API switch case does not call break. (Varun Thacker via shalin) + +* SOLR-6882: Misspelled collection API actions in ReplicaMutator exception messages. + (Steve Rowe via shalin) + +* SOLR-6867: SolrCLI should check for existence before creating a new core/collection, + more user-friendly error reporting (no stack trace), and the ability to pass a + directory when using bin/solr to create a core or collection (Timothy Potter) + +* SOLR-6885: Add core name to RecoveryThread name. (Christine Poerschke via shalin) + +* SOLR-6855: bin/solr -e dih launches, but has some path cruft issues preventing some of the + imports don't work (Hossman, Timothy Potter) + +* SOLR-3711: Truncate long strings in /browse field facets (ehatcher) + +* SOLR-6876: Remove unused legacy scripts.conf (Alexandre Rafalovitch via Erick Erickson) + +* SOLR-6896: Speed up tests by dropping SolrJettyRunner thread max idle time + (Alan Woodward) + +* SOLR-6448: Add SolrJ support for all current Collection API calls. (Anshum Gupta) + +* Fixed a typo in various solrconfig.xml files. (sdumitriu - pull request #120) + +* SOLR-6895: SolrServer classes are renamed to *SolrClient. The existing + classes still exist, but are deprecated. (Alan Woodward, Erik Hatcher) + +* SOLR-6483: Refactor some methods in MiniSolrCloudCluster tests (Steve Davids via + Erick Erickson) + +* SOLR-6906: Fix typo bug in DistributedDebugComponentTest.testCompareWithNonDistributedRequest + (Ramkumar Aiyengar via Erick Erickson) + +* SOLR-6905: Test pseudo-field retrieval in distributed search. + (Ramkumar Aiyengar via shalin) + +* SOLR-6897: Nuke non-NRT mode from code and configuration. (Hossman, shalin) + +* SOLR-6830: Update Woodstox to 4.4.1 and StAX to 3.1.4. (ab) + +* SOLR-6918: No need to log exceptions (as warn) generated when creating MBean stats if + the core is shutting down (Timothy Potter) + +* SOLR-6932: All HttpClient ConnectionManagers and SolrJ clients should always be shutdown + in tests and regular code. (Mark Miller) + +* SOLR-1723: VelocityResponseWriter improvements (Erik Hatcher) + +* SOLR-6324: Set finite default timeouts for select and update. (Ramkumar Aiyengar via Mark Miller) + +* SOLR-6952: bin/solr create action should copy configset directory instead of reusing + an existing configset in ZooKeeper by default (Timothy Potter) + +* SOLR-6933: bin/solr should provide a single "create" action that creates a core + or collection depending on whether Solr is running in standalone or cloud mode + (Timothy Potter) + +* SOLR-6496: LBHttpSolrClient stops server retries after the timeAllowed threshold is met. + (Steve Davids, Anshum Gupta) + +* SOLR-6904: Removed deprecated Circle & rect syntax. See upgrading notes. (David Smiley) + +* SOLR-6943: HdfsDirectoryFactory should fall back to system props for most of it's config + if it is not found in solrconfig.xml. (Mark Miller, Mike Drob) + +* SOLR-6926: "ant example" makes no sense anymore - should be "ant server" + (Ramkumar Aiyengar, Timothy Potter) + +* SOLR-6982: bin/solr and SolrCLI should support SSL-related Java System Properties + (Timothy Potter) + +* SOLR-6981: Add a delete action to the bin/solr script to allow deleting of cores / + collections (with delete collection config directory from ZK) (Timothy Potter) + +* SOLR-6840: Remove support for old-style solr.xml (Erick Erickson, Alan Woodward) + +* SOLR-6976: Remove classes and methods deprecated in 4.x (Alan Woodward, Noble + Paul, Chris Hostetter) + +* SOLR-6521: CloudSolrClient should synchronize cache cluster state loading + ( Noble Paul, Jessica Cheng Mallet) + +* SOLR-7018: bin/solr stop should stop if there is only one node running or generate + an error message prompting the user to be explicit about which of multiple nodes + to stop using the -p or -all options (Timothy Potter) + +* SOLR-5918: ant clean does not remove ZooKeeper data (Varun Thacker, Steve Rowe) + +* SOLR-7020: 'bin/solr start' should automatically use an SSL-enabled alternate jetty + configuration file when in SSL mode, eliminating the need for manual jetty.xml edits. + (Steve Rowe) + +* SOLR-6227: Avoid spurious failures of ChaosMonkeySafeLeaderTest by ensuring there's + at least one jetty to kill. (shalin) + +================== 4.10.4 ================== + +Bug Fixes +---------------------- + +* SOLR-6931: We should do a limited retry when using HttpClient. + (Mark Miller, Hrishikesh Gadre, Gregory Chanan) + +* SOLR-6780: Fixed a bug in how default/appends/invariants params were affecting the set + of all "keys" found in the request parameters, resulting in some key=value param pairs + being duplicated. This was noticeably affecting some areas of the code where iteration + was done over the set of all params: + - literal.* in ExtractingRequestHandler + - facet.* in FacetComponent + - spellcheck.[dictionary name].* and spellcheck.collateParam.* in SpellCheckComponent + - olap.* in AnalyticsComponent + (Alexandre Rafalovitch & hossman) + +* SOLR-6426: SolrZkClient clean can fail due to a race with children nodes. (Mark Miller) + +* SOLR-6457: LBHttpSolrClient: ArrayIndexOutOfBoundsException risk if counter overflows + (longkey via Noble Paul) + +* SOLR-6481: CLUSTERSTATUS should check if the node hosting a replica is live when + reporting replica status (Timothy Potter) + +* SOLR-6631: DistributedQueue spinning on calling zookeeper getChildren() + (Jessica Cheng Mallet, Mark Miller, Timothy Potter) + +* SOLR-6579: SnapPuller Replication blocks clean shutdown of tomcat + (Philip Black-Knight via Noble Paul) + +* SOLR-6763: Shard leader elections should not persist across session expiry + (Alan Woodward, Mark Miller) + +* SOLR-3881: Avoid OOMs in LanguageIdentifierUpdateProcessor: + - Added langid.maxFieldValueChars and langid.maxTotalChars params to limit + input, by default 10k and 20k chars, respectively. + - Moved input concatenation to Tika implementation; the langdetect + implementation instead appends each input piece via the langdetect API. + (Vitaliy Zhovtyuk, Tomás Fernández Löbbe, Rob Tulloh, Steve Rowe) + +* SOLR-6850: AutoAddReplicas makes a call to wait to see live replicas that times + out after 30 milliseconds instead of 30 seconds. (Varun Thacker via Mark Miller) + +* SOLR-6839: Direct routing with CloudSolrServer will ignore the Overwrite document option. + (Mark Miller) + +* SOLR-7139: Fix SolrContentHandler for TIKA to ignore multiple startDocument events. + (Chris A. Mattmann, Uwe Schindler) + +* SOLR-6941: DistributedQueue#containsTaskWithRequestId can fail with NPE. (Mark Miller) + +* SOLR-7011: Delete collection returns before collection is actually removed. + (Christine Poerschke via shalin) + +* SOLR-6856: Restore ExtractingRequestHandler's ability to capture all HTML tags when + parsing (X)HTML. (hossman, Uwe Schindler, ehatcher, Steve Rowe) + +* SOLR-6928: solr.cmd stop works only in english (john.work, Jan Høydahl, Timothy Potter) + +* SOLR-7038: Validate the presence of configset before trying to create a collection. + (Anshum Gupta, Mark Miller) + +* SOLR-7016: Fix bin\solr.cmd to work in a directory with spaces in the name. + (Timothy Potter, Uwe Schindler) + +* SOLR-6693: bin\solr.cmd doesn't support 32-bit JRE/JDK running on Windows due to + parenthesis in JAVA_HOME. (Timothy Potter, Christopher Hewitt, Jan Høydahl) + +* SOLR-7067: bin/solr won't run under bash 4.2+. (Steve Rowe) + +* SOLR-7033, SOLR-5961: RecoveryStrategy should not publish any state when + closed / cancelled and there should always be a pause between recoveries + even when recoveries are rapidly stopped and started as well as when a + node attempts to become the leader for a shard. + (Mark Miller, Maxim Novikov) + +* SOLR-6847: LeaderInitiatedRecoveryThread compares wrong replica's state with lirState. + (shalin) + +* SOLR-7128: Two phase distributed search is fetching extra fields in GET_TOP_IDS phase. + (Pablo Queixalos, shalin) + +Other Changes +---------------------- + +* SOLR-7147: Introduce new TrackingShardHandlerFactory for monitoring what requests + are sent to shards during tests. (hossman, shalin) + +================== 4.10.3 ================== + +Bug Fixes +---------------------- + +* SOLR-6696: bin/solr start script should not enable autoSoftCommit by default (janhoy) + +* SOLR-6704: TrieDateField type drops schema properties in branch 4.10 (Tomás Fernández Löbbe) + +* SOLR-6085: Suggester crashes when prefixToken is longer than surface form (janhoy) + +* SOLR-6323: ReRankingQParserPlugin cleaner paging and fix bug with fuzzy, range and other queries + that need to be re-written. (Adair Kovac, Joel Bernstein) + +* SOLR-6684: Fix-up /export JSON. (Joel Bernstein) + +* SOLR-6781: BBoxField didn't support dynamic fields. (David Smiley) + +* SOLR-6784: BBoxField's 'score' mode should have been optional. (David Smiley) + +* SOLR-6510: The collapse QParser would throw a NPE when used on a DocValues field on + an empty segment/index. (Christine Poerschke, David Smiley) + +* SOLR-2927: Solr does not unregister all mbeans upon exception in constructor + causing memory leaks. (tom liu, Sharath Babu, Cyrille Roy, shalin) + +* SOLR-6685: ConcurrentModificationException in Overseer Status API. (shalin) + +* SOLR-6706: /update/json/docs throws RuntimeException if a nested structure + contains a non-leaf float field (Noble Paul, shalin) + +* SOLR-6610: Slow startup of new clusters because ZkController.publishAndWaitForDownStates + always times out. (Jessica Cheng Mallet, shalin, Noble Paul) + +* SOLR-6662: better validation when parsing command-line options that expect a value + (Timothy Potter) + +* SOLR-6732: Fix handling of leader-initiated recovery state was String in older versions + and is now a JSON map, caused backwards compatibility issues when doing rolling upgrades of + a live cluster while indexing (Timothy Potter) + +* SOLR-6705: Better strategy for dealing with JVM specific options in the start + scripts; remove -XX:+AggressiveOpts and only set -XX:-UseSuperWord for Java 1.7u40 + to u51. (Uwe Schindler, janhoy, hossman, Timothy Potter) + +* SOLR-6726: better strategy for selecting the JMX RMI port based on SOLR_PORT in bin/solr + script (Timothy Potter) + +* SOLR-6795: distrib.singlePass returns score even though not asked for. + (Per Steffensen via shalin) + +* SOLR-6796: distrib.singlePass does not return correct set of fields for multi-fl-parameter + requests. (Per Steffensen via shalin) + +* SOLR-6776: Transaction log was not flushed at the end of update requests with softCommit + specified, which could lead to data loss if the server were killed immediately after the + update finished. (Jeffery Yuan via yonik) + +Other Changes +---------------------- + +* SOLR-6661: Adjust all example configurations to allow overriding error-prone + relative paths for solrconfig.xml <lib> references with solr.install.dir + system property; bin/solr scripts will set it appropriately. (ehatcher) + +* SOLR-6694: Auto-detect JAVA_HOME using the Windows registry if it is not set + (janhoy, Timothy Potter) + +* SOLR-6653: bin/solr script should return error code >0 when something fails + (janhoy, Timothy Potter) + +* SOLR-6829: Added getter/setter for lastException in DIH's ContextImpl (ehatcher) + +================== 4.10.2 ================== + +Bug Fixes +---------------------- + +* SOLR-6509: Solr start scripts interactive mode doesn't honor -z argument (Timothy Potter) + +* SOLR-6511: Fencepost error in LeaderInitiatedRecoveryThread (Timothy Potter) + +* SOLR-6530: Commits under network partitions can put any node in down state. + (Ramkumar Aiyengar, Alan Woodward, Mark Miller, shalin) + +* SOLR-6573: QueryElevationComponent now works with localParams in the query (janhoy) + +* SOLR-6524: Collections left in recovery state after node restart because recovery sleep time + increases exponentially between retries. (Mark Miller, shalin) + +* SOLR-6587: Misleading exception when creating collections in SolrCloud with bad configuration. + (Tomás Fernández Löbbe) + +* SOLR-6452: StatsComponent's stat 'missing' will work on fields with docValues=true and + indexed=false (Xu Zhang via Tomás Fernández Löbbe) + +* SOLR-6646: bin/solr start script fails to detect solr on non-default port and then after + 30s tails wrong log file (janhoy) + +* SOLR-6647: Bad error message when missing resource from ZK when parsing Schema (janhoy) + +* SOLR-6545: Query field list with wild card on dynamic field fails. + (Burke Webster, Xu Zhang, shalin) + +Other Changes +---------------------- + +* SOLR-6550: Provide simple mechanism for passing additional metadata / context about a server-side + SolrException back to the client-side (Timothy Potter) + +* SOLR-6486: solr start script can have a debug flag option; use -a to set arbitrary options + (Noble Paul, Timothy Potter) + +* SOLR-6549: bin/solr script should support a -s option to set the -Dsolr.solr.home property. + (Timothy Potter) + +* SOLR-6529: Stop command in the start scripts should only stop the instance that it had started. + (Varun Thacker, Timothy Potter) + +================== 4.10.1 ================== + +Bug Fixes +---------------------- + +* SOLR-6425: If using the new global hdfs block cache option, you can end up + reading corrupt files on file name reuse. (Mark Miller, Gregory Chanan) + +* SOLR-5814: CoreContainer reports incorrect & misleading path for solrconfig.xml + when there are loading problems (Pradeep via hossman) + +* SOLR-6024: Fix StatsComponent when using docValues="true" multiValued="true" + (Vitaliy Zhovtyuk & Tomas Fernandez-Lobbe via hossman) + +* SOLR-6493: Fix fq exclusion via "ex" local param in multivalued stats.field (hossman) + +* SOLR-6447: bin/solr script needs to pass -DnumShards=1 for boostrapping collection1 + when starting Solr in cloud mode. (Timothy Potter) + +* SOLR-6501: Binary Response Writer does not return wildcard fields. + (Mike Hugo, Constantin Mitocaru, sarowe, shalin) + +Other Changes +--------------------- + +* SOLR-6503: Removed support for parsing netcdf files in Solr Cell because + of license issues. If you need support for this format, download the parser + JAR yourself (version 4.2) and add it to contrib/extraction/lib folder: + http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/ + (Uwe Schindler) + +================== 4.10.0 ================= + +Consult the LUCENE_CHANGES.txt file for additional, low level, changes in this release + +Versions of Major Components +--------------------- +Apache Tika 1.5 (with upgraded Apache POI 3.10.1) +Carrot2 3.9.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 + +Upgrading from Solr 4.9 +---------------------- + +* In Solr 3.6, all primitive field types were changed to omit norms by default when the + schema version is 1.5 or greater (SOLR-3140), but TrieDateField's default was mistakenly + not changed. As of Solr 4.10, TrieDateField omits norms by default (see SOLR-6211). + +* Creating a SolrCore via CoreContainer.create() no longer requires an + additional call to CoreContainer.register() to make it available to clients + (see SOLR-6170). + +* CoreContainer.remove() has been removed. You should now use CoreContainer.unload() to + delete a SolrCore (see SOLR-6232). + +* solr.xml parsing has been improved to better account for the expected data types of + various options. As part of this fix, additional error checking has also been added to + provide errors in the event of duplicated options, or unknown option names that may + indicate a typo. Users who have modified their solr.xml in the past and now upgrade may + get errors on startup if they have typos or unexpected options specified in their solr.xml + file. (See SOLR-5746 for more information.) + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-6196: The overseerstatus collection API instruments amILeader and ZK state update calls. + (shalin) + +* SOLR-6069: The 'clusterstatus' API should return 'roles' information. (shalin) + +* SOLR-6044: The 'clusterstatus' API should return live_nodes as well. (shalin) + +* SOLR-5768: Add a distrib.singlePass parameter to make EXECUTE_QUERY phase fetch all fields + and skip GET_FIELDS. (Gregg Donovan, shalin) + +* SOLR-6183: New spatial BBoxField for indexing rectangles with search support for most predicates. + It includes extra score relevancy modes in addition to distance: score=overlapRatio|area|area2D. + (David Smiley, Ryan McKinley) + +* SOLR-6232: You can now unload/delete cores that have failed to initialize (Alan Woodward) + +* SOLR-2245: Improvements to the MailEntityProcessor: + - Support for server-side date filtering if using GMail; requires new + dependency on the Sun Gmail Java mail extensions + - Support for using the last_index_time from the previous run as the + value for the fetchMailsSince filter. + (Peter Sturge, Timothy Potter) + +* SOLR-6258: Added onRollback event handler hook to Data Import Handler (DIH). + (ehatcher) + +* SOLR-6263: Add DIH handler name to variable resolver as ${dih.handlerName}. (ehatcher) + +* SOLR-6216: Better faceting for multiple intervals on DV fields (Tomas Fernandez-Lobbe + via Erick Erickson) + +* SOLR-6267: Let user override Interval Faceting key with LocalParams (Tomas Fernandez_Lobbe + via Erick Erickson) + +* SOLR-6020: Auto-generate a unique key in schema-less example if data does not have an id field. + The UUIDUpdateProcessor was improved to not require a field name in configuration and generate + a UUID into the unique Key field. + (Vitaliy Zhovtyuk, hossman, Steve Rowe, Erik Hatcher, shalin) + +* SOLR-6294: SOLR-6437: Remove the restriction of adding json by only wrapping it in an array in a + new path /update/json/docs (Noble Paul , hossman, Yonik Seeley, Steve Rowe) + +* SOLR-6302: UpdateRequestHandlers are registered implicitly /update , + /update/json, /update/csv , /update/json/docs (Noble Paul) + +* SOLR-6318: New "terms" QParser for efficiently filtering documents by a list of values. For + many values, it's more appropriate than a boolean query. (David Smiley) + +* SOLR-6283: Add support for Interval Faceting in SolrJ. (Tomás Fernández Löbbe) + +* SOLR-6304 : JsonLoader should be able to flatten an input JSON to multiple docs (Noble Paul) + +* SOLR-2894: Distributed query support for facet.pivot (Dan Cooper, Erik Hatcher, Chris Russell, + Andrew Muldowney, Brett Lucey, Mark Miller, hossman) + +* SOLR-5656: Add autoAddReplicas feature for shared file systems. (Mark Miller, Gregory Chanan) + +* SOLR-5244: Exporting Full Sorted Result Sets (Erik Hatcher, Joel Bernstein) + +* SOLR-3617: bin/solr and bin/solr.cmd scripts for starting, stopping, and running Solr examples + (Timothy Potter) + +* SOLR-6233: Provide basic command line tools for checking Solr status and health. + (Timothy Potter) + + +Bug Fixes +---------------------- + +* SOLR-6095 : SolrCloud cluster can end up without an overseer with overseer roles (Noble Paul, Shalin Mangar) + +* SOLR-6165: DataImportHandler should write BigInteger and BigDecimal values as strings. + (Anand Sengamalai via shalin) + +* SOLR-6189: Avoid publishing the state as down if the node is not live when determining + if a replica should be in leader-initiated recovery. (Timothy Potter) + +* SOLR-6197: The MIGRATE collection API doesn't work when legacyCloud=false is set + in cluster properties. (shalin) + +* SOLR-6206: The migrate collection API fails on retry if temp collection already exists. + (shalin) + +* SOLR-6072: The 'deletereplica' API should remove the data and instance directory by default. + (shalin) + +* SOLR-6211: TrieDateField doesn't default to omitNorms=true. (Michael Ryan, Steve Rowe) + +* SOLR-6159: A ZooKeeper session expiry during setup can keep LeaderElector from joining elections. + (Steven Bower, shalin) + +* SOLR-6223: SearchComponents may throw NPE when using shards.tolerant and there is a failure + in the 'GET_FIELDS/GET_HIGHLIGHTS/GET_DEBUG' phase. (Tomás Fernández Löbbe via shalin) + +* SOLR-6180: Callers of ManagedIndexSchema mutators should hold the schemaUpdateLock. + (Gregory Chanan via Steve Rowe) + +* SOLR-6229: Make SuggestComponent return 400 instead of 500 for bad dictionary selected in request. + (Tomás Fernández Löbbe via shalin) + +* SOLR-6235: Leader initiated recovery should use coreNodeName instead of coreName to avoid marking + all replicas having common core name as down. (shalin) + +* SOLR-6208: JettySolrRunner QueuedThreadPool's configuration code is never executed. (dweiss via shalin) + +* SOLR-6245: Socket and Connection configuration are ignored in HttpSolrServer when passing in HttpClient. + (Patanachai Tangchaisin, shalin) + +* SOLR-6137: Schemaless concurrency improvements: + - Fixed an NPE when reloading a managed schema with no dynamic copy fields + - Moved parsing and schema fields addition to after the distributed phase + - AddSchemaFieldsUpdateProcessor now uses a fixed schema rather than always + retrieving the latest, and holds the schema update lock through the entire + schema swap-out process + (Gregory Chanan via Steve Rowe) + +* SOLR-6136: ConcurrentUpdateSolrServer includes a Spin Lock (Brandon Chapman, Timothy Potter) + +* SOLR-6257: More than two "!"-s in a doc ID throws an + ArrayIndexOutOfBoundsException when using the composite id router. + (Steve Rowe) + +* SOLR-5746: Bugs in solr.xml parsing have been fixed to more correctly deal with the various + datatypes of options people can specify, additional error handling of duplicated/unidentified + options has also been added. (Maciej Zasada, hossman) + +* SOLR-5847: Fixed data import abort button in admin UI. (ehatcher) + +* SOLR-6264: Distributed commit and optimize are executed serially across all + replicas. (Mark Miller, Timothy Potter) + +* SOLR-6163: Correctly decode special characters in managed stopwords and synonym endpoints. + (Vitaliy Zhovtyuk, Timo Schmidt via Timothy Potter) + +* SOLR-6336: DistributedQueue can easily create too many ZooKeeper Watches. + (Ramkumar Aiyengar via Mark Miller) + +* SOLR-6347: DELETEREPLICA throws a NPE while removing the last Replica in a Custom + sharded collection. (Anshum Gupta) + +* SOLR-6062: Fix undesirable edismax query parser effect (introduced in SOLR-2058) in how phrase queries + generated from pf, pf2, and pf3 are merged into the main query. (Michael Dodsworth via ehatcher) + +* SOLR-6372: HdfsDirectoryFactory should use supplied Configuration for communicating with secure kerberos. + (Gregory Chanan via Mark Miller) + +* SOLR-6284: Fix NPE in OCP when non-existent sliceId is used for a + deleteShard request (Ramkumar Aiyengar via Anshum Gupta) + +* SOLR-6380: Added missing context info to log message if IOException occurs in processing tlog + (Steven Bower via hossman) + +* SOLR-6383: RegexTransformer returns no results after replaceAll if regex does not match a value. + (Alexander Kingson, shalin) + +* SOLR-6387: Add better error messages throughout Solr and supply a work around for + Java bug #8047340 to SystemInfoHandler: On Turkish default locale, some JVMs fail + to fork on MacOSX, BSD, AIX, and Solaris platforms. (hossman, Uwe Schindler) + +* SOLR-6338: coreRootDirectory requires trailing slash, or SolrCloud cores are created in wrong location. + (Primož Skale via Erick Erickson) + +* SOLR-6314: Facet counts duplicated in the response if specified more than once on the request. + (Vamsee Yarlagadda, Erick Erickson) + +* SOLR-6378: Fixed example/example-DIH/ issues with "tika" and "solr" configurations, and tidied up README.txt + (Daniel Shchyokin via ehatcher) + +* SOLR-6393: TransactionLog replay performance on HDFS is very poor. (Mark Miller) + +* SOLR-6268: HdfsUpdateLog has a race condition that can expose a closed HDFS FileSystem instance and should + close its FileSystem instance if either inherited close method is called. (Mark Miller) + +* SOLR-6089: When using the HDFS block cache, when a file is deleted, its underlying data entries in the + block cache are not removed, which is a problem with the global block cache option. + (Mark Miller, Patrick Hunt) + +* SOLR-6402: OverseerCollectionProcessor should not exit for ZooKeeper ConnectionLoss. + (Jessica Cheng via Mark Miller) + +* SOLR-6405: ZooKeeper calls can easily not be retried enough on ConnectionLoss. + (Jessica Cheng, Mark Miller) + +* SOLR-6410: Ensure all Lookup instances are closed via CloseHook + (hossman, Areek Zillur, Ryan Ernst, Dawid Weiss) + +Optimizations +--------------------- + +* LUCENE-5803: Solr's schema now uses DelegatingAnalyzerWrapper. This uses less heap + for cached TokenStreamComponents because it caches per FieldType not per Field, so + indexes with many fields of same type just use one TokenStream per thread. + (Shay Banon, Uwe Schindler, Robert Muir) + +* SOLR-6259: Reduce CPU usage by avoiding repeated costly calls to Document.getField inside + DocumentBuilder.toDocument for use-cases with large number of fields and copyFields. + (Steven Bower via shalin) + +* SOLR-5968: BinaryResponseWriter fetches unnecessary stored fields when only pseudo-fields + are requested. (Gregg Donovan via shalin) + +* SOLR-6261: Run ZooKeeper watch event callbacks in parallel to the ZooKeeper + event thread. (Ramkumar Aiyengar via Mark Miller) + +Other Changes +--------------------- + +* SOLR-6173: Fixed wrong failure message in TestDistributedSearch. (shalin) + +* SOLR-5902: Corecontainer level mbeans are not exposed (noble) + +* SOLR-6194: Allow access to DataImporter and DIHConfiguration from DataImportHandler. + (Aaron LaBella via shalin) + +* SOLR-6170: CoreContainer.preRegisterInZk() and CoreContainer.register() commands + are merged into CoreContainer.create(). (Alan Woodward) + +* SOLR-6171: Remove unused SolrCores coreNameToOrig map (Alan Woodward) + +* SOLR-5596: Set system property zookeeper.forceSync=no for Solr test cases. (shalin) + +* SOLR-2853: Add a unit test for the case when "spellcheck.maxCollationTries=0" (James Dyer) + +* SOLR-6240: Removed unused coreName parameter in ZkStateReader.getReplicaProps. (shalin) + +* SOLR-6241: Harden the HttpPartitionTest. (shalin) + +* SOLR-6228: Fixed bug in TestReplicationHandler.doTestIndexAndConfigReplication. (shalin) + +* SOLR-6120: On Windows, when the war is not extracted, the zkcli.bat script + will print a helpful message indicating that the war must be unzipped instead + of a java error about a missing class. (shalin, Shawn Heisey) + +* SOLR-6179: Better strategy for handling empty managed data to avoid spurious + warning messages in the logs. (Timothy Potter) + +* SOLR-6232: CoreContainer.remove() replaced with CoreContainer.unload(). A call to + unload will also close the core. + +* SOLR-3893: DIH should not depend on mail.jar,activation.jar (Timothy Potter, Steve Rowe) + +* SOLR-6252: A couple of small improvements to UnInvertedField class. + (Vamsee Yarlagadda, Gregory Chanan, Mark Miller) + +* SOLR-3345: BaseDistributedSearchTestCase should always ignore QTime. + (Vamsee Yarlagadda, Benson Margulies via Mark Miller) + +* SOLR-6270: Increased timeouts for MultiThreadedOCPTest. (shalin) + +* SOLR-6274: UpdateShardHandler should log the params used to configure its + HttpClient. (Ramkumar Aiyengar via Mark Miller) + +* SOLR-6194: Opened up "public" access to DataSource, DocBuilder, and EntityProcessorWrapper + in DIH. (Aaron LaBella via ehatcher) + +* SOLR-6269: Renamed "rollback" to "error" in DIH internals, including renaming onRollback + to onError introduced in SOLR-6258. (ehatcher) + +* SOLR-3622: When using DIH in SolrCloud-mode, rollback will no longer be called when + an error occurs. (ehatcher) + +* SOLR-6231: Increased timeouts and hardened the RollingRestartTest. (Noble Paul, shalin) + +* SOLR-6290: Harden and speed up CollectionsAPIAsyncDistributedZkTest. (Mark Miller, shalin) + +* SOLR-6281: Made PostingsSolrHighlighter more configurable via subclass extension. (David Smiley) + +* SOLR-6309: Increase timeouts for AsyncMigrateRouteKeyTest. (shalin) + +* SOLR-2168: Added support for facet.missing in /browse field and pivot faceting. (ehatcher) + +* SOLR-4702: Added support for multiple spellcheck collations to /browse UI. (ehatcher) + +* SOLR-5664: Added support for multi-valued field highlighting in /browse UI. (ehatcher) + +* SOLR-6313: Improve SolrCloud cloud-dev scripts. (Mark Miller, Vamsee Yarlagadda) + +* SOLR-6360: Remove bogus "Content-Charset" header in HttpSolrServer. (Michael Ryan, + Uwe Schindler) + +* SOLR-6362: Fix bug in TestSqlEntityProcessorDelta. (James Dyer) + +* SOLR-6388: Force upgrade of Apache POI dependency in Solr Cell to version + 3.10.1 to fix CVE-2014-3529 and CVE-2014-3574. (Uwe Schindler) + +* SOLR-6391: Improve message for CREATECOLLECTION failure due to missing + numShards (Anshum Gupta) + +================== 4.9.1 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.5 (with upgraded Apache POI 3.10.1) +Carrot2 3.9.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 + +Detailed Change List +---------------------- + +Other Changes +--------------------- + +* SOLR-6503: Removed support for parsing netcdf files in Solr Cell because + of license issues. If you need support for this format, download the parser + JAR yourself (version 4.2) and add it to contrib/extraction/lib folder: + http://www.unidata.ucar.edu/software/thredds/current/netcdf-java/ + (Uwe Schindler) + +* SOLR-6388: Force upgrade of Apache POI dependency in Solr Cell to version + 3.10.1 to fix CVE-2014-3529 and CVE-2014-3574. (Uwe Schindler) + +================== 4.9.0 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.5 +Carrot2 3.9.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 + +Upgrading from Solr 4.8 +---------------------- + +* Support for DiskDocValuesFormat (ie: fieldTypes configured with docValuesFormat="Disk") + has been removed due to poor performance. If you have an existing fieldTypes using + DiskDocValuesFormat please modify your schema.xml to remove the 'docValuesFormat' + attribute, and optimize your index to rewrite it into the default codec, prior to + upgrading to 4.9. See LUCENE-5761 for more details. + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-5999: Add checkIntegrityAtMerge support to solrconfig.xml. + (Varun Thacker via Ryan Ernst) + +* SOLR-6043: Add ability to set http headers in solr response + (Tomás Fernández Löbbe via Ryan Ernst) + +* SOLR-5973: Pluggable Ranking Collectors and Merge Strategies + (Joel Bernstein) + +* SOLR-6108: Add support for 'addreplica' Collection API in SolrJ. (shalin) + +* SOLR-5468: Allow a client application to request the minium achieved + replication factor for an update request (single or batch) by sending + an optional parameter "min_rf". (Timothy Potter) + +* SOLR-6088: Add query re-ranking with the ReRankingQParserPlugin + (Joel Bernstein) + +* SOLR-5285: Added a new [child ...] DocTransformer for optionally including + Block-Join descendant documents inline in the results of a search. This works + independent of whether the search itself is a block-join related query and is + supported by he xml, json, and javabin response formats. + (Varun Thacker via hossman) + +* SOLR-6150: Add new AnalyticsQuery to support pluggable analytics + (Joel Bernstein) + +* SOLR-6125: Allow SolrIndexWriter to close without waiting for merges + (Christine Poerschke via Alan Woodward) + +* SOLR-6064: DebugComponent track output should be returned as a JSON + object rather than a list (Christine Poerschke, Alan Woodward) + + +Bug Fixes +---------------------- + +* SOLR-5956: Use coreDescriptor.getInstanceDir() instead of getRawInstanceDir() + in the SnapShooter to avoid problems when solr.solr.home is a symbolic link. + (Timothy Potter) + +* SOLR-6002: Fix a couple of ugly issues around SolrIndexWriter close and + rollback as well as how SolrIndexWriter manages its ref counted directory + instance. (Mark Miller, Gregory Chanan) + +* SOLR-6015: Better way to handle managed synonyms when ignoreCase=true + (Timothy Potter) + +* SOLR-6104: The 'addreplica' Collection API does not support 'async' parameter. + (shalin) + +* SOLR-6101: Shard splitting doesn't work when legacyCloud=false is set in + cluster properties. (shalin) + +* SOLR-6111: The 'deleteshard' collection API should be able to delete a shard + in 'construction' state. (shalin) + +* SOLR-6118: 'expand.sort' didn't support function queries. (David Smiley) + +* SOLR-6120: zkcli.sh should expand solr.war automatically instead of throwing + ClassNotFoundException. (sebastian badea, shalin) + +* SOLR-6149: Specifying the query value without any index value does not work in + Analysis browser. (Aman Tandon, shalin) + +* SOLR-6145: Fix Schema API optimistic concurrency by moving it out of + ManagedIndexSchema.add(Copy)Fields() into the consumers of those methods: + CopyFieldCollectionResource, FieldCollectionResource, FieldResource, + and AddSchemaFieldsUpdateProcessorFactory. + (Gregory Chanan, Alexey Serba, Steve Rowe) + +* SOLR-6146: Incorrect configuration such as wrong chroot in zk server address can + cause CloudSolrServer to leak resources. (Jessica Cheng, Varun Thacker, shalin) + +* SOLR-6158: Relative configSetBase directories were resolved relative to the + container CWD, rather than solr.home. (Simon Endele, Alan Woodward) + +* SOLR-5426: Fixed a bug in ReverseWildCardFilter that could cause + InvalidTokenOffsetsException when highlighting. (Uwe Schindler, Arun Kumar, via hossman) + +* SOLR-6175: DebugComponent throws NPE on shard exceptions when using shards.tolerant. + (Tomás Fernández Löbbe via shalin) + +* SOLR-6129: DateFormatTransformer doesn't resolve dateTimeFormat. (Aaron LaBella via shalin) + +* SOLR-6164: Copy Fields Schema additions are not distributed to other nodes. + (Gregory Chanan via Steve Rowe) + +* SOLR-6160: An error was sometimes possible if a distributed search included grouping + with group.facet, faceting on facet.field and either facet.range or facet.query. + (David Smiley) + +* SOLR-6182: Data stored by the RestManager could not be reloaded after core restart, causing + the core to fail to load; cast the data loaded from storage to the correct data type. + (Timothy Potter) + +Other Changes +--------------------- + +* SOLR-5980: AbstractFullDistribZkTestBase#compareResults always returns false + for shouldFail. (Mark Miller, Gregory Chanan) + +* SOLR-5987: Add "collection" to UpdateParams. (Mark Miller, Greg Solovyev) + +* SOLR-3862: Add remove" as update option for atomically removing a value + from a multivalued field (Jim Musli, Steven Bower, Alaknantha via Erick Erickson) + +* SOLR-5974: Remove ShardDoc.score and use parent's ScoreDoc.score. + (Tomás Fernández Löbbe via Ryan Ernst) + +* SOLR-6025: Replace mentions of CommonsHttpSolrServer with HttpSolrServer and + StreamingUpdateSolrServer with ConcurrentUpdateSolrServer. (Ahmet Arslan via shalin) + +* SOLR-6013: Fix method visibility of Evaluator, refactor DateFormatEvaluator for + extensibility. (Aaron LaBella via shalin) + +* SOLR-6022: Deprecate getAnalyzer() in IndexField and FieldType, and add getIndexAnalyzer(). + (Ryan Ernst) + +* SOLR-3671: Fix DIHWriter interface usage so users may implement writers that output + documents to a location external to Solr (ex. a NoSql db). (Roman Chyla via James Dyer) + +* SOLR-5340: Add support for named snapshots (Varun Thacker via Noble Paul) + +* SOLR-5495: Recovery strategy for leader partitioned from replica case. Hardening + recovery scenarios after the leader receives an error trying to forward an + update request to a replica. (Timothy Potter) + +* SOLR-6116: Refactor DocRouter.getDocRouter to accept routerName as a String. (shalin) + +* SOLR-6026: REQUESTSTATUS Collection API now also checks for submitted tasks which are + yet to begin execution. + +* SOLR-6067: Refactor duplicate Collector code in SolrIndexSearcher + (Christine Poerschke via hossman) + +* SOLR-5940: post.jar reports back detailed error in case of error responses. + (Sameer Maggon, shalin, Uwe Schindler) + +* SOLR-6161: SolrDispatchFilter should throw java.lang.Error back even if wrapped in + another exception. (Miklos Christine via shalin) + +* SOLR-6153: ReplicationHandler backup response format should contain backup name. + (Varun Thacker via shalin) + +* SOLR-6169: Remove broken handleAlias action in CoreAdminHandler (Alan + Woodward) + +* SOLR-6128: Removed deprecated analysis factories and fieldTypes from the example + schema.xml (hossman) + +* SOLR-5868: HttpClient should be configured to use ALLOW_ALL_HOSTNAME hostname + verifier to simplify SSL setup. (Steve Davids via Mark Miller) + + +Optimizations +---------------------- + +* SOLR-5681: Make the processing of Collection API calls multi-threaded. + (Anshum Gupta, shalin, Noble Paul) + +Build +--------------------- + +* SOLR-6006: Separate test and compile scope dependencies in the Solrj and + Solr contrib ivy.xml files, so that the derived Maven dependencies get + filled out properly in the corresponding POMs. (Steven Scott, Steve Rowe) + +* SOLR-6130: Added com.uwyn:jhighlight dependency to, and removed asm:asm + dependency from the extraction contrib - dependencies weren't fully + upgraded with the Tika 1.4->1.5 upgrade (SOLR-5763). (Steve Rowe) + +================== 4.8.1 ================== + +Bug Fixes +---------------------- + +* SOLR-5904: ElectionContext can cancel an election when it should not if there + was an exception while trying to register as the leader. + (Mark Miller, Alan Woodward) + +* SOLR-5993: ZkController can warn about shard leader conflict even after the conflict + is resolved. (Gregory Chanan via shalin) + +* SOLR-6017: Fix SimpleQParser to use query analyzer + (Ryan Ernst) + +* SOLR-6029: CollapsingQParserPlugin throws ArrayIndexOutOfBoundsException + if elevated doc has been deleted from a segment. (Greg Harris, Joel Bernstein) + +* SOLR-6030: Use System.nanoTime() instead of currentTimeInMills() in LRUCache.warm. + (Tomás Fernández Löbbe via shalin) + +* SOLR-6037: Fixed incorrect max/sum/stddev for Date fields in StatsComponent + (Brett Lucey, hossman) + +* SOLR-6023: FieldAnalysisRequestHandler throws NPE if no parameters are supplied. + (shalin) + +* SOLR-5090: SpellCheckComponent sometimes throws NPE if + "spellcheck.alternativeTermCount" is set to zero (James Dyer). + +* SOLR-6039: fixed debug output when no results in response + (Tomás Fernández Löbbe, hossman) + +* SOLR-6035: CloudSolrServer directUpdate routing should use getCoreUrl. + (Marvin Justice, Joel Bernstein) + +================== 4.8.0 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.5 +Carrot2 3.9.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.6 + +Upgrading from Solr 4.7 +---------------------- + +* In previous versions of Solr, Terms that exceeded Lucene's MAX_TERM_LENGTH were + silently ignored when indexing documents. Beginning with Solr 4.8, a document + an error will be generated when attempting to index a document with a term + that is too large. If you wish to continue to have large terms ignored, + use "solr.LengthFilterFactory" in all of your Analyzers. See LUCENE-5472 for + more details. + +* Solr 4.8 requires Java 7 or greater, Java 8 is verified to be + compatible and may bring some performance improvements. When using + Oracle Java 7 or OpenJDK 7, be sure to not use the GA build 147 or + update versions u40, u45 and u51! We recommend using u55 or later. + An overview of known JVM bugs can be found on + http://wiki.apache.org/lucene-java/JavaBugs + +* ZooKeeper is upgraded from 3.4.5 to 3.4.6. + +* <fields> and <types> tags have been deprecated. There is no longer any reason to + keep them in the schema file, they may be safely removed. This allows intermixing of + <fieldType>, <field> and <copyField> definitions if desired. Currently, these tags + are supported so either style may be implemented. TBD is whether they'll be + deprecated formally for 5.0 + + +Detailed Change List +---------------------- + +System Requirements +---------------------- + +* LUCENE-4747, LUCENE-5514: Move to Java 7 as minimum Java version. + (Robert Muir, Uwe Schindler) + +New Features +---------------------- + +* SOLR-5130: Implement addReplica Collections API (Noble Paul) + +* SOLR-5183: JSON updates now support nested child documents using a + "_childDocument_" object key. (Varun Thacker, hossman) + +* SOLR-5714: You can now use one pool of memory for for the HDFS block cache + that all collections share. (Mark Miller, Gregory Chanan) + +* SOLR-5720: Add ExpandComponent to expand results collapsed by the + CollapsingQParserPlugin. (Joel Bernstein) + +* SOLR-3177: Enable tagging and excluding filters in StatsComponent via the + localParams syntax. (Mathias H., Nikolai Luthman, Vitaliy Zhovtyuk, shalin) + +* SOLR-1604: Wildcards, ORs etc inside Phrase Queries. (Ahmet Arslan via Erick Erickson) + +* SOLR-5477: Async execution of OverseerCollectionProcessor(CollectionsAPI) + tasks. (Anshum Gupta) + +* SOLR-5865: Provide a MiniSolrCloudCluster to enable easier testing. + (Greg Chanan via Mark Miller) + +* SOLR-5860: Use leaderConflictResolveWait in WaitForState during recovery/startup, + improve logging and force refresh cluster state every 15 seconds. + (Timothy Potter via shalin) + +* SOLR-5749: A new Overseer status collection API exposes overseer queue sizes, timing + statistics, success and error counts and last N failures per operation. (shalin) + +* SOLR-5858: Add a hl.qparser parameter to allow you to define a queryparser + for hl.q highlight queries. If no queryparser is defined, Solr will use + the overall query's defType. (Alan Woodward) + +* SOLR-4478: Allow cores to use configuration from a configsets directory + outside their instance directory. (Alan Woodward, Erick Erickson) + +* SOLR-5466: A new List collections and cluster status API which clients can use + to read collection and shard information instead of reading data directly from ZooKeeper. + (Dave Seltzer, Varun Thacker, Vitaliy Zhovtyuk, Erick Erickson, shalin) + +* SOLR-5795: New DocExpirationUpdateProcessorFactory supports computing an expiration + date for documents from the "TTL" expression, as well as automatically deleting expired + documents on a periodic basis. (hossman) + +* SOLR-5829: Allow ExpandComponent to accept query and filter query parameters + (Joel Bernstein) + +* SOLR-5653: Create a RestManager to provide REST API endpoints for + reconfigurable plugins. (Tim Potter, Steve Rowe) + +* SOLR-5655: Create a stopword filter factory that is (re)configurable, + and capable of reporting its configuration, via REST API. + (Tim Potter via Steve Rowe) + +* SOLR-5654: Create a synonym filter factory that is (re)configurable, and + capable of reporting its configuration, via REST API. + (Tim Potter via Steve Rowe) + +* SOLR-5960: Add support for basic authentication in post.jar tool, e.g.: + java -Durl="http://username:password@hostname:8983/solr/update" -jar post.jar sample.xml + (Sameer Maggon via Uwe Schindler) + +* SOLR-4864: RegexReplaceProcessorFactory should support pattern capture group + substitution in replacement string. + (Sunil Srinivasan, Jack Krupansky via Steve Rowe) + +Bug Fixes +---------------------- + +* SOLR-5858, SOLR-4812: edismax and dismax query parsers can be used for parsing + highlight queries. (Alan Woodward, Tien Nguyen Manh) + +* SOLR-5893: On restarting overseer designate , move itself to front of the queue (Noble Paul) + +* SOLR-5915: Attempts to specify the parserImpl for + solr.PreAnalyzedField fieldtype failed. (Mike McCandless) + +* SOLR-5943: SolrCmdDistributor does not distribute the openSearcher parameter. + (ludovic Boutros via shalin) + +* SOLR-5954: Slower DataImportHandler process caused by not reusing jdbc + connections. (Mark Miller, Paco Garcia, Raja Nagendra Kumar) + +* SOLR-5897: Upgraded to jQuery 1.7.2, Solr was previously using 1.4.3, the file was + mistakenly named 1.7.2 (steffkes) + +Optimizations +---------------------- +* SOLR-1880: Distributed Search skips GET_FIELDS stage if EXECUTE_QUERY + stage gets all fields. Requests with fl=id or fl=id,score are now single-pass. + (Shawn Smith, Vitaliy Zhovtyuk, shalin) + +* SOLR-5783: Requests to open a new searcher will now reuse the current registered + searcher (w/o additional warming) if possible in situations where the underlying + index has not changed. This reduces overhead in situations such as deletes that + do not modify the index, and/or redundant commits. (hossman) + +* SOLR-5884: When recovery is cancelled, any call to the leader to wait to see + the replica in the right state for recovery should be aborted. (Mark Miller) + +Other Changes +--------------------- + +* SOLR-5909: Upgrade Carrot2 clustering dependency to 3.9.0. (Dawid Weiss) + +* SOLR-5764: Fix recently added tests to not use absolute paths to load test-files, + use SolrTestCaseJ4.getFile() and getResource() instead; fix morphlines/map-reduce + to not duplicate test resources and fix dependencies among them. + (Uwe Schindler) + +* SOLR-5765: Update to SLF4J 1.7.6. (Mark Miller) + +* SOLR-5609: If legacy mode is disabled don't let cores create slices/replicas/collections . + All operations should be performed through collection API (Noble Paul) + +* SOLR-5613: Upgrade to commons-codec 1.9 for better BeiderMorseFilter performance. + (Thomas Champagne, Shawn Heisey via shalin) + +* SOLR-5771: Add SolrTestCaseJ4.SuppressSSL annotation to disable SSL (instead of static boolean). + (Robert Muir) + +* SOLR-5799: When registering as the leader, if an existing ephemeral + registration exists, wait a short time to see if it goes away. + (Mark Miller) + +* LUCENE-5472: IndexWriter.addDocument will now throw an IllegalArgumentException + if a Term to be indexed exceeds IndexWriter.MAX_TERM_LENGTH. To recreate previous + behavior of silently ignoring these terms, use LengthFilter in your Analyzer. + (hossman, Mike McCandless, Varun Thacker) + +* SOLR-5825: Separate http request creating and execution in SolrJ + (Steven Bower via Erick Erickson) + +* SOLR-5837: Add hashCode/equals to SolrDocument, SolrInputDocument + and SolrInputField for testing purposes. (Varun Thacker, Noble Paul, + Mark Miller) + +* SOLR-5853: The createCollection methods in the test framework now reports + result of operation in the returned CollectionAdminResponse (janhoy) + +* SOLR-5838: Relative SolrHome Path Bug At AbstractFullDistribZkTestBase. + (Furkan KAMACI via shalin) + +* SOLR-5763: Upgrade to Tika 1.5 (Vitaliy Zhovtyuk via Steve Rowe) + +* SOLR-5881: Upgrade ZooKeeper to 3.4.6 (Shawn Heisey) + +* SOLR-5883: Many tests do not shutdown SolrServer. + (Tomás Fernández Löbbe via Mark Miller) + +* SOLR-5898: Update to latest Kite Morphlines release: Version 0.12.1. + (Mark Miller) + +* SOLR-5228: Don't require <field> or <dynamicField> be inside of <fields> -- or + that <fieldType> be inside of <types>. (Erick Erickson) + +* SOLR-5903: SolrCore implements Closeable, cut over to using try-with-resources + where possible. (Alan Woodward) + +* SOLR-5914: Cleanup and fix Solr's test cleanup code. + (Mark Miller, Uwe Schindler) + +* SOLR-5936: Deprecate non-Trie-based numeric & date field types. (Steve Rowe) + +* SOLR-5934: LBHttpSolrServer exception handling improvement and small test + improvements. (Gregory Chanan via Mark Miller) + +* SOLR-5773: CollapsingQParserPlugin should make elevated documents the + group head. (David Boychuck, Joel Bernstein) + +* SOLR-5937: Modernize the DIH example config sets. (Steve Rowe) + +================== 4.7.2 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.4 +Carrot2 3.8.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.5 + +Detailed Change List +---------------------- + +Bug Fixes +---------------------- + +* SOLR-5951: Fixed SolrDispatchFilter to throw useful exception on startup if + SLF4j logging jars are missing. (Uwe Schindler, Hossman, Shawn Heisey) + +* SOLR-5950: Maven config: make the org.slf4j:slf4j-api dependency transitive + (i.e., not optional) in all modules in which it's a dependency, including + solrj, except for the WAR, where it will remain optional. + (Uwe Schindler, Steve Rowe) + +================== 4.7.1 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.4 +Carrot2 3.8.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.5 + +Detailed Change List +---------------------- + +Bug Fixes +---------------------- + +* SOLR-5647: The lib paths in example-schemaless will now load correctly. + (Paul Westin via Shawn Heisey) + +* SOLR-5770: All attempts to match a SolrCore with its state in clusterstate.json + should be done with the CoreNodeName. (Steve Davids via Mark Miller) + +* SOLR-5875: QueryComponent.mergeIds() unmarshals all docs' sort field values once + per doc instead of once per shard. + (Alexey Serba, hoss, Martin de Vries via Steve Rowe) + +* SOLR-5800: Admin UI - Analysis form doesn't render results correctly when a + CharFilter is used. (steffkes) + +* SOLR-5870: Admin UI - Reload on Core Admin doesn't show errors (steffkes) + +* SOLR-5867: OverseerCollectionProcessor isn't properly generating https urls in some + cases. (Steve Davids via shalin) + +* SOLR-5866: UpdateShardHandler needs to use the system default scheme registry to + properly handle https via javax.net.ssl.* properties. (Steve Davids via shalin) + +* SOLR-5782: The full MapReduceIndexer help text does not display when using --help. + (Mark Miller, Wolfgang Hoschek) + +* SOLR-5824: Merge up Solr MapReduce contrib code to latest external changes. + Includes a few minor bug fixes. + (Mark Miller) + +* SOLR-5818: distrib search with custom comparator does not quite work correctly + (Ryan Ernst) + +* SOLR-5895: JavaBinLoader hides IOExceptions. (Mike Sokolov via shalin) + +* SOLR-5861: Recovery should not set onlyIfLeaderActive=true for slice in 'recovery' + state. (shalin) + +* SOLR-5423: CSV output doesn't include function field + (Arun Kumar, hossman, Steve Rowe) + +* SOLR-5550: shards.info is not returned by a short circuited distributed query. + (Timothy Potter, shalin) + +* SOLR-5777: Fix ordering of field values in JSON updates where + field name key is repeated (hossman) + +* SOLR-5734: We should use System.nanoTime rather than System.currentTimeMillis + when calculating elapsed time. (Mark Miller, Ramkumar Aiyengar) + +* SOLR-5760: ConcurrentUpdateSolrServer has a blockUntilFinished call when + streamDeletes is true that should be tucked into the if statement below it. + (Mark Miller, Gregory Chanan) + +* SOLR-5761: HttpSolrServer has a few fields that can be set via setters but + are not volatile. (Mark Miller, Gregory Chanan) + +* SOLR-5907: The hdfs write cache can cause a reader to see a corrupted state. + It now defaults to off, and if you were using solr.hdfs.blockcache.write.enabled + explicitly, you should set it to false. + (Mark Miller) + +* SOLR-5811: The Overseer will retry work items until success, which is a serious + problem if you hit a bad work item. (Mark Miller) + +* SOLR-5796: Increase how long we are willing to wait for a core to see the ZK + advertised leader in its local state. (Timothy Potter, Mark Miller) + +* SOLR-5834: Overseer threads are only being interrupted and not closed. + (hossman, Mark Miller) + +* SOLR-5839: ZookeeperInfoServlet does not trim path properly. + (Furkan KAMACI via Mark Miller) + +* SOLR-5874: Unsafe cast in CloudSolrServer's RouteException. Change + RouteException to handle Throwable rather than Exception. + (Mark Miller, David Arthur) + +* SOLR-5899: CloudSolrServer's RouteResponse and RouteException should be + publicly accessible. (Mark Miller, shalin) + +* SOLR-5905: CollapsingQParserPlugin throws a NPE if required 'field' param is missing. + (Spyros Kapnissis via shalin) + +* SOLR-5906: Collection create API ignores property.instanceDir parameter. + (Varun Thacker, shalin) + +* SOLR-5920: Distributed sort on DateField, BoolField and BCD{Int,Long,Str}Field + returns string cast exception (Eric Bus, AJ Lemke, hossman, Steve Rowe) + +Other Changes +--------------------- + +* SOLR-5796: Make how long we are willing to wait for a core to see the ZK + advertised leader in its local state configurable. + (Timothy Potter via Mark Miller) + +================== 4.7.0 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.4 +Carrot2 3.8.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.5 + +Upgrading from Solr 4.6.0 +---------------------- + +* CloudSolrServer and LBHttpSolrServer no longer declare MalformedURLException + as thrown from their constructors. + +* Due to a bug in previous versions the default value of the 'discountOverlap' property + of DefaultSimilarity was not being set appropriately if you were using the implicit + DefaultSimilarityFactory instead of explicitly configuring it. To preserve + consistent behavior for people who upgrade, the implicit behavior is now contingent + on the <luceneMatchVersion/> -- discountOverlap=false for 4.6 and below, + discountOverlap=true for 4.7 and above. See SOLR-5561 for more information. + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-5308: SOLR-5601: SOLR-5710: A new 'migrate' collection API to split all + documents with a route key into another collection (shalin) + +* SOLR-5441: Expose number of transaction log files and their size via JMX. + (Rafał Kuć via shalin) + +* SOLR-5320: Added support for tri-level compositeId routing. + (Anshum Gupta via shalin) + +* SOLR-5458: Admin UI - Added a new "Files" conf directory browser/file viewer. + (steffkes) + +* SOLR-5447, SOLR-5490: Add a QParserPlugin for Lucene's SimpleQueryParser. + (Jack Conradson via shalin) + +* SOLR-5208: Support for the setting of core.properties key/values at create-time on + Collections API (Erick Erickson) + +* SOLR-5428: SOLR-5690: New 'stats.calcdistinct' parameter in StatsComponent returns + set of distinct values and their count. This can also be specified per field + e.g. 'f.field.stats.calcdistinct'. (Elran Dvir via shalin) + +* SOLR-5378, SOLR-5528: A new SuggestComponent that fully utilizes the Lucene suggester + module and adds pluggable dictionaries, payloads and better distributed support. + This is intended to eventually replace the Suggester support through the + SpellCheckComponent. (Areek Zillur, Varun Thacker via shalin) + +* SOLR-5492: Return the replica that actually served the query in shards.info + response. (shalin) + +* SOLR-5506: Support docValues in CollationField and ICUCollationField. + (Robert Muir) + +* SOLR-5023: Add support for deleteInstanceDir to be passed from SolrJ for Core + Unload action. (Lyubov Romanchuk, shalin) + +* SOLR-1871: The 'map' function query accepts a ValueSource as target and + default value. (Chris Harris, shalin) + +* SOLR-5556: Allow class of CollectionsHandler and InfoHandler to be specified + in solr.xml. (Gregory Chanan, Alan Woodward) + +* SOLR-5581: Give ZkCLI the ability to get files. (Gregory Chanan via Mark Miller) + +* SOLR-5536: Add ValueSource collapse criteria to CollapsingQParsingPlugin (Joel Bernstein) + +* SOLR-5541: Allow QueryElevationComponent to accept elevateIds and excludeIds + as http parameters (Joel Bernstein) + +* SOLR-5463: new 'cursorMark' request param for deep paging of sorted result sets + (sarowe, hossman) + +* SOLR-5529: Add support for queries to use multiple suggesters. + (Areek Zillur, Erick Erickson, via Robert Muir) + +* SOLR-1301: Add a Solr contrib that allows for building Solr indexes via + Hadoop's MapReduce. (Matt Revelle, Alexander Kanarsky, Steve Rowe, + Mark Miller, Greg Bowyer, Jason Rutherglen, Kris Jirapinyo, Jason Venner , + Andrzej Bialecki, Patrick Hunt, Wolfgang Hoschek, Roman Shaposhnik, + Eric Wong) + +* SOLR-5631: Add support for Lucene's FreeTextSuggester. + (Areek Zillur via Robert Muir) + +* SOLR-5695: Add support for Lucene's BlendedInfixSuggester. + (Areek Zillur) + +* SOLR-5476: Overseer Role for nodes (Noble Paul) + +* SOLR-5594: Allow FieldTypes to specify custom PrefixQuery behavior + (Anshum Gupta via hossman) + +* LUCENE-5395: Upgrade to Spatial4j 0.4. Various new options are now exposed + automatically for an RPT field type. See Spatial4j CHANGES & javadocs. + https://github.com/spatial4j/spatial4j/blob/master/CHANGES.md (David Smiley) + +* SOLR-5670: allow _version_ to use DocValues. (Per Steffensen via yonik) + +* SOLR-5535: Set "partialResults" header for shards that error out if + shards.tolerant is specified. (Steve Davids via shalin) + +* SOLR-5610: Support cluster-wide properties with an API called CLUSTERPROP (Noble Paul) + +* SOLR-5623: Better diagnosis of RuntimeExceptions in analysis + (Benson Margulies) + +* SOLR-5530: Added a NoOpResponseParser for SolrJ which puts the entire raw + response into an entry in the NamedList. + (Upayavira, Vitaliy Zhovtyuk via shalin) + +* SOLR-5682: Make the admin InfoHandler more pluggable / derivable. + (Greg Chanan via Mark Miller) + +* SOLR-5672: Add logParamsList parameter to support reduced logging. + (Christine Poerschke via Mark Miller) + +* SOLR-3854: SSL support for SolrCloud. (Sami Siren, hossman, Steve Davids, + Alexey Serba, Mark Miller) + +Bug Fixes +---------------------- + +* SOLR-5438: DebugComponent throws NPE when used with grouping. + (Tomás Fernández Löbbe via shalin) + +* SOLR-4612: Admin UI - Analysis Screen contains empty table-columns (steffkes) + +* SOLR-5451: SyncStrategy closes its http connection manager before the + executor that uses it in its close method. (Mark Miller) + +* SOLR-5460: SolrDispatchFilter#sendError can get a SolrCore that it does not + close. (Mark Miller) + +* SOLR-5461: Request proxying should only set con.setDoOutput(true) if the + request is a post. (Mark Miller) + +* SOLR-5481: SolrCmdDistributor should not let the http client do its own + retries. (Mark Miller) + +* LUCENE-5347: Fixed Solr's Zookeeper Client to copy files to Zookeeper using + binary transfer. Previously data was read with default encoding and stored + in zookeeper as UTF-8. This bug was found after upgrading to forbidden-apis + 1.4. (Uwe Schindler) + +* SOLR-4376: DataImportHandler uses wrong date format for last_index_time if + a delta-import is run first before any full-imports. + (Sebastien Lorber, Arcadius Ahouansou via shalin) + +* SOLR-5494: CoreContainer#remove throws NPE rather than returning null when + a SolrCore does not exist in core discovery mode. (Mark Miller) + +* SOLR-5354: Distributed sort is broken with CUSTOM FieldType. + (Steve Rowe, hossman, Robert Muir, Jessica Cheng) + +* SOLR-5515: NPE when getting stats on date field with empty result on + SolrCloud. (Alexander Sagen, shalin) + +* SOLR-5204: StatsComponent and SpellCheckComponent do not support the + shards.tolerant=true parameter. (Anca Kopetz, shalin) + +* SOLR-5527: DIH logs spurious warning for special commands. (shalin) + +* SOLR-5524: Exception when using Query Function inside Scale Function. + (Trey Grainger, yonik) + +* SOLR-5562: ConcurrentUpdateSolrServer constructor ignores supplied httpclient. + (Kyle Halliday via Mark Miller) + +* SOLR-5567: ZkController getHostAddress duplicates url prefix. + (Kyle Halliday, Alexey Serba, shalin) + +* SOLR-4992: Solr eats OutOfMemoryError exceptions in many cases. + (Mark Miller, Daniel Collins) + +* LUCENE-5399, SOLR-5354 sort wouldn't work correctly with + distributed searching for some field types such as legacy numeric + types (Rob Muir, Mike McCandless) + +* SOLR-5643: ConcurrentUpdateSolrServer will sometimes not spawn a new Runner + thread even though there are updates in the queue. (Mark Miller) + +* SOLR-5650: When a replica becomes a leader, only peer sync with other replicas + that last published an ACTIVE state. (Mark Miller) + +* SOLR-5657: When a SolrCore starts on HDFS, it should gracefully handle HDFS + being in safe mode. (Mark Miller) + +* SOLR-5663: example-DIH uses non-existing column for mapping (case-sensitive) + (steffkes) + +* SOLR-5666: Using the hdfs write cache can result in appearance of corrupted + index. (Mark Miller) + +* SOLR-5230: Call DelegatingCollector.finish() during grouping. + (Joel Bernstein, ehatcher) + +* SOLR-5679: Shard splitting fails with ClassCastException on collections + upgraded from 4.5 and earlier versions. (Brett Hoerner, shalin) + +* SOLR-5673: HTTPSolrServer doesn't set own property correctly in + setFollowRedirects. (Frank Wesemann via shalin) + +* SOLR-5676: SolrCloud updates rejected if talking to secure ZooKeeper. + (Greg Chanan via Mark Miller) + +* SOLR-5634: SolrJ GroupCommand.getNGroups returns null if group.format=simple + and group.ngroups=true. (Artem Lukanin via shalin) + +* SOLR-5667: Performance problem when not using hdfs block cache. (Mark Miller) + +* SOLR-5526: Fixed NPE that could arise when explicitly configuring some built + in QParserPlugins (Nikolay Khitrin, Vitaliy Zhovtyuk, hossman) + +* SOLR-5598: LanguageIdentifierUpdateProcessor ignores all but the first value + of multiValued string fields. (Andreas Hubold, Vitaliy Zhovtyuk via shalin) + +* SOLR-5593: Replicas should accept the last updates from a leader that has just + lost its connection to ZooKeeper. (Christine Poerschke via Mark Miller) + +* SOLR-5678: SolrZkClient should throw a SolrException when connect times out + rather than a RuntimeException. (Karl Wright, Anshum Gupta, Mark Miller) + +* SOLR-4072: Error message is incorrect for linkconfig in ZkCLI. + (Vamsee Yarlagadda, Adam Hahn, via Mark Miller) + +* SOLR-5691: Sharing non thread safe WeakHashMap across thread can cause + problems. (Bojan Smid, Mark Miller) + +* SOLR-5693: Running on HDFS does work correctly with NRT search. (Mark Miller) + +* SOLR-5644: SplitShard does not handle not finding a shard leader well. + (Mark Miller, Anshum Gupta via shalin) + +* SOLR-5704: coreRootDirectory was not respected when creating new cores + via CoreAdminHandler (Jesse Sipprell, Alan Woodward) + +* SOLR-5709: Highlighting grouped duplicate docs from different shards with + group.limit > 1 throws ArrayIndexOutOfBoundsException. (Steve Rowe) + +* SOLR-5561: Fix implicit DefaultSimilarityFactory initialization in IndexSchema + to properly specify discountOverlap option. + (Isaac Hebsh, Ahmet Arslan, Vitaliy Zhovtyuk, hossman) + +* SOLR-5689: On reconnect, ZkController cancels election on first context rather + than latest. (Gregory Chanan, Mark Miller via shalin) + +* SOLR-5649: Clean up some minor ConnectionManager issues. + (Mark Miller, Gregory Chanan) + +* SOLR-5365: Fix bug with compressed files in ExtractingRequestHandler by + upgrading commons-compress to 1.7 (Jan Høydahl, hossman) + +* SOLR-5675: cloud-scripts/zkcli.bat: quote option log4j + (Günther Ruck via steffkes + +* SOLR-5721: ConnectionManager can become stuck in likeExpired. + (Gregory Chanan via Mark Miller) + +* SOLR-5731: In ConnectionManager, we should catch and only log exceptions + from BeforeReconnect. (Mark Miller) + +* SOLR-5718: Make LBHttpSolrServer zombie checks non-distrib and non-scoring. + (Christine Poerschke via Mark Miller) + +* SOLR-5727: LBHttpSolrServer should only retry on Connection exceptions when + sending updates. Affects CloudSolrServer. (Mark Miller) + +* SOLR-5739: Sub-shards created by shard splitting have their update log set + to buffering mode on restarts. (Günther Ruck, shalin) + +* SOLR-5741: UpdateShardHandler was not correctly setting max total connections + on the HttpClient. (Shawn Heisey) + +* SOLR-5620: ZKStateReader.aliases should be volatile to ensure all threads see + the latest aliases. (Ramkumar Aiyengar via Mark Miller) + +* SOLR-5448: ShowFileRequestHandler treats everything as Directory, when in + Cloud-Mode. (Erick Erickson, steffkes) + +Optimizations +---------------------- + +* SOLR-5436: Eliminate the 1500ms wait in overseer loop as well as + polling the ZK distributed queue. (Noble Paul, Mark Miller) + +* SOLR-5189: Solr 4.x Web UI Log Viewer does not display 'date' column from + logs (steffkes) + +* SOLR-5512: Optimize DocValuesFacets. (Robert Muir) + +* SOLR-2960: fix DIH XPathEntityProcessor to add the correct number of "null" + placeholders for multi-valued fields (Michael Watts via James Dyer) + +* SOLR-5214: Reduce memory usage for shard splitting by merging segments one + at a time. (Christine Poerschke via shalin) + +* SOLR-4227: Wrap XML RequestWriter's OutputStreamWriter in a BufferedWriter + to avoid frequent converter invocations. (Conrad Herrmann, shalin) + +* SOLR-5624: Enable QueryResultCache for CollapsingQParserPlugin. + (David Boychuck, Joel Bernstein) + +* LUCENE-5440: DocSet decoupled from OpenBitSet. DocSetBase moved to use + FixedBitSet instead of OpenBitSet. As a result BitDocSet now only works + with FixedBitSet. (Shai Erera) + +Other Changes +--------------------- + +* SOLR-5399: Add distributed request tracking information to DebugComponent + (Tomás Fernández Löbbe via Ryan Ernst) + +* SOLR-5421: Remove double set of distrib.from param in processAdd method of + DistributedUpdateProcessor. (Anshum Gupta via shalin) + +* SOLR-5404: The example config references deprecated classes. + (Uwe Schindler, Rafał Kuć via Mark Miller) + +* SOLR-5487: Replication factor error message doesn't match constraint. + (Patrick Hunt via shalin) + +* SOLR-5499: Log a warning if /get is not registered when using SolrCloud. + (Daniel Collins via shalin) + +* SOLR-5517: Return HTTP error on POST requests with no Content-Type. + (Ryan Ernst, Uwe Schindler) + +* SOLR-5502: Added a test for tri-level compositeId routing with documents + having a "/" in a document id. (Anshum Gupta via Mark Miller) + +* SOLR-5533: Improve out of the box support for running Solr on hdfs with + SolrCloud. (Mark Miller) + +* SOLR-5548: Give DistributedSearchTestCase / JettySolrRunner the ability to + specify extra filters. (Greg Chanan via Mark Miller) + +* SOLR-5555: LBHttpSolrServer and CloudSolrServer constructors don't need to + declare MalformedURLExceptions (Sushil Bajracharya, Alan Woodward) + +* SOLR-5565: Raise default ZooKeeper session timeout to 30 seconds from 15 + seconds. (Mark Miller) + +* SOLR-5574: CoreContainer shutdown publishes all nodes as down and waits to + see that and then again publishes all nodes as down. (Mark Miller) + +* SOLR-5590: Upgrade HttpClient/HttpComponents to 4.3.x. + (Karl Wright via Shawn Heisey) + +* SOLR-2794: change the default of hl.phraseLimit to 5000. + (Michael Della Bitta via Robert Muir, Koji, zarni - pull request #11) + +* SOLR-5632: Improve response message for reloading a non-existent core. + (Anshum Gupta via Mark Miller) + +* SOLR-5633: HttpShardHandlerFactory should make its http client available to subclasses. + (Ryan Ernst) + +* SOLR-5684: Shutdown SolrServer clients created in BasicDistributedZk2Test and + BasicDistributedZkTest. (Tomás Fernández Löbbe via shalin) + +* SOLR-5629: SolrIndexSearcher.name should include core name. + (Shikhar Bhushan via shalin) + +* SOLR-5702: Log config name found for collection at info level. + (Christine Poerschke via Mark Miller) + +* SOLR-5659: Add test for compositeId ending with an '!'. + (Markus Jelsma, Anshum Gupta via shalin) + +* SOLR-5700: Improve error handling of remote queries (proxied requests). + (Greg Chanan, Steve Davids via Mark Miller) + +* SOLR-5585: Raise Collections API timeout to 3 minutes from one minute. + (Mark Miller) + +* SOLR-5257: Improved error/warn messages when Update XML contains unexpected XML nodes + (Vitaliy Zhovtyuk, hossman) + + +================== 4.6.1 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.4 +Carrot2 3.8.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.5 + +Detailed Change List +---------------------- + +Bug Fixes +---------------------- + +* SOLR-5408: CollapsingQParserPlugin scores incorrectly when multiple sort criteria are used + (Brandon Chapman, Joel Bernstein) + +* SOLR-5416: CollapsingQParserPlugin breaks Tag/Exclude Faceting (David Boychuck, Joel Bernstein) + +* SOLR-5442: Python client cannot parse proxied response when served by Tomcat. + (Patrick Hunt, Gregory Chanan, Vamsee Yarlagadda, Romain Rigaux, Mark Miller) + +* SOLR-5445: Proxied responses should propagate all headers rather than the + first one for each key. (Patrick Hunt, Mark Miller) + +* SOLR-5479: SolrCmdDistributor retry logic stops if a leader for the request + cannot be found in 1 second. (Mark Miller) + +* SOLR-5532: SolrJ Content-Type validation is too strict for some + webcontainers / proxies. (Jakob Furrer, hossman, Shawn Heisey, Uwe Schindler, + Mark Miller) + +* SOLR-5547: Creating a collection alias using SolrJ's CollectionAdminRequest + sets the alias name and the collections to alias to the same value. + (Aaron Schram, Mark Miller) + +* SOLR-5577: Likely ZooKeeper expiration should not slow down updates a given + amount, but instead cut off updates after a given time. + (Mark Miller, Christine Poerschke, Ramkumar Aiyengar) + +* SOLR-5580: NPE when creating a core with both explicit shard and coreNodeName. + (YouPeng Yang, Mark Miller) + +* SOLR-5552: Leader recovery process can select the wrong leader if all replicas + for a shard are down and trying to recover as well as lose updates that should + have been recovered. (Timothy Potter, Mark Miller) + +* SOLR-5569 A replica should not try and recover from a leader until it has + published that it is ACTIVE. (Mark Miller) + +* SOLR-5568 A SolrCore cannot decide to be the leader just because the cluster + state says no other SolrCore's are active. (Mark Miller) + +* SOLR-5496: We should share an http connection manager across non search + HttpClients and ensure all http connection managers get shutdown. + (Mark Miller) + +* SOLR-5583: ConcurrentUpdateSolrServer#blockUntilFinished may wait forever if + the executor service is shutdown. (Mark Miller) + +* SOLR-5586: All ZkCmdExecutor's should be initialized with the zk client + timeout. (Mark Miller) + +* SOLR-5587: ElectionContext implementations should use + ZkCmdExecutor#ensureExists to ensure their election paths are properly + created. (Mark Miller) + +* SOLR-5540: HdfsLockFactory should explicitly create the lock parent directory if + necessary. (Mark Miller) + +* SOLR-4709: The core reload after replication if config files have changed + can fail due to a race condition. (Mark Miller, Hossman) + +* SOLR-5503: Retry 'forward to leader' requests less aggressively - rather + than on IOException and status 500, ConnectException. (Mark Miller) + +* SOLR-5588: PeerSync doesn't count all connect failures as success. + (Mark Miller) + +* SOLR-5564: hl.maxAlternateFieldLength should apply to original field when + fallback is attempted (janhoy) + +* SOLR-5608: Don't allow a closed SolrCore to publish state to ZooKeeper. + (Mark Miller, Shawn Heisey) + +* SOLR-5615: Deadlock while trying to recover after a ZK session expiration. + (Ramkumar Aiyengar, Mark Miller) + +* SOLR-5543: Core swaps resulted in duplicate core entries in solr.xml when + using solr.xml persistence. (Bill Bell, Alan Woodward) + +* SOLR-5618: Fix false cache hits in queryResultCache when hashCodes are equal + and duplicate filter queries exist in one of the requests (hossman) + +* SOLR-4260: ConcurrentUpdateSolrServer#blockUntilFinished can return before + all previously added updates have finished. This could cause distributed + updates meant for replicas to be lost. (Markus Jelsma, Timothy Potter, + Joel Bernstein, Mark Miller) + +* SOLR-5645: A SolrCore reload via the CoreContainer will try and register in + zk again with the new SolrCore. (Mark Miller) + +* SOLR-5636: SolrRequestParsers does some xpath lookups on every request, which + can cause concurrency issues. (Mark Miller) + +* SOLR-5658: commitWithin and overwrite are not being distributed to replicas + now that SolrCloud uses javabin to distribute updates. + (Mark Miller, Varun Thacker, Elodie Sannier, shalin) + +Optimizations +---------------------- + +* SOLR-5576: Improve concurrency when registering and waiting for all + SolrCore's to register a DOWN state. (Christine Poerschke via Mark Miller) + +================== 4.6.0 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.4 +Carrot2 3.8.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.5 + +Upgrading from Solr 4.5.0 +---------------------- + +* If you are using methods from FieldMutatingUpdateProcessorFactory for getting + configuration information (oneOrMany or getBooleanArg), those methods have + been moved to NamedList and renamed to removeConfigArgs and removeBooleanArg, + respectively. The original methods are deprecated, to be removed in 5.0. + See SOLR-5264. + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-5167: Add support for AnalyzingInfixSuggester (AnalyzingInfixLookupFactory). + (Areek Zillur, Varun Thacker via Robert Muir) + +* SOLR-5246: Shard splitting now supports collections configured with router.field. + (shalin) + +* SOLR-5274: Allow JettySolrRunner SSL config to be specified via a constructor. + (Mark Miller) + +* SOLR-5300: Shards can be split by specifying arbitrary number of hash ranges + within the shard's hash range. (shalin) + +* SOLR-5226: Add Lucene index heap usage to the Solr admin UI. + (Areek Zillur via Robert Muir) + +* SOLR-5324: Make sub shard replica recovery and shard state switch asynchronous. + (Yago Riveiro, shalin) + +* SOLR-5338: Split shards by a route key using split.key parameter. (shalin) + +* SOLR-5353: Enhance CoreAdmin api to split a route key's documents from an index + and leave behind all other documents. (shalin) + +* SOLR-5027: CollapsingQParserPlugin for high performance field collapsing on high cardinality fields. + (Joel Bernstein) + +* SOLR-5395: Added a RunAlways marker interface for UpdateRequestProcessorFactory + implementations indicating that they should not be removed in later stages + of distributed updates (usually signalled by the update.distrib parameter) + (yonik) + + * SOLR-5310: Add a collection admin command to remove a replica (noble) + + * SOLR-5311: Avoid registering replicas which are removed (noble) + + * SOLR-5406: CloudSolrServer failed to propagate request parameters + along with delete updates. (yonik) + + * SOLR-5374: Support user configured doc-centric versioning rules + via the optional DocBasedVersionConstraintsProcessorFactory + update processor (Hossman, yonik) + +* SOLR-5392: Extend solrj apis to cover collection management. + (Roman Shaposhnik via Mark Miller) + +* SOLR-5084: new field type EnumField. (Elran Dvir via Erick Erickson) + +* SOLR-5464: Add option to ConcurrentSolrServer to stream pure delete + requests. (Mark Miller) + +Bug Fixes +---------------------- + +* SOLR-5216: Document updates to SolrCloud can cause a distributed deadlock. + (Mark Miller) + +* SOLR-5367: Unmarshalling delete by id commands with JavaBin can lead to class cast + exception. (Mark Miller) + +* SOLR-5359: ZooKeeper client is not closed when it fails to connect to an ensemble. + (Mark Miller, Klaus Herrmann) + +* SOLR-5042: MoreLikeThisComponent was using the rows/count value in place of + flags, which caused a number of very strange issues, including NPEs and + ignoring requests for the results to include the score. + (Anshum Gupta, Mark Miller, Shawn Heisey) + +* SOLR-5371: Solr should consistently call SolrServer#shutdown (Mark Miller) + +* SOLR-5363: Solr doesn't start up properly with Log4J2 (Petar Tahchiev via Alan + Woodward) + +* SOLR-5380: Using cloudSolrServer.setDefaultCollection(collectionId) does not + work as intended for an alias spanning more than 1 collection. + (Thomas Egense, Shawn Heisey, Mark Miller) + +* SOLR-5418: Background merge after field removed from solr.xml causes error. + (Reported on user's list, Robert M's patch via Erick Erickson) + +* SOLR-5318: Creating a core via the admin API doesn't respect transient property + (Olivier Soyez via Erick Erickson) + +* SOLR-5388: Creating a new core via the HTTP API that results in a transient being + unloaded results in a " Too many close [count:-1]" error. + (Olivier Soyez via Erick Erickson) + +* SOLR-5453: Raise recovery socket read timeouts. (Mark Miller) + +* SOLR-5397: Replication can fail silently in some cases. (Mark Miller) + +* SOLR-5465: SolrCmdDistributor retry logic has a concurrency race bug. + (Mark Miller) + +* SOLR-5452: Do not attempt to proxy internal update requests. (Mark Miller) + +Optimizations +---------------------- + +* SOLR-5232: SolrCloud should distribute updates via streaming rather than buffering. + (Mark Miller) + +* SOLR-5223: SolrCloud should use the JavaBin binary format for communication by default. + (Mark Miller) + +* SOLR-5370: Requests to recover when an update fails should be done in + background threads. (Mark Miller) + +* LUCENE-5300,LUCENE-5304: Specialized faceting for fields which are declared as + multi-valued in the schema but are actually single-valued. (Adrien Grand) + +Security +---------------------- + +* SOLR-4882: SolrResourceLoader was restricted to only allow access to resource + files below the instance dir. The reason for this is security related: Some + Solr components allow to pass in resource paths via REST parameters + (e.g. XSL stylesheets, velocity templates,...) and load them via resource + loader. For backwards compatibility, this security feature can be disabled + by a new system property: solr.allow.unsafe.resourceloading=true + (Uwe Schindler) + +Other Changes +---------------------- + +* SOLR-5237: Add indexHeapUsageBytes to LukeRequestHandler, indicating how much + heap memory is being used by the underlying Lucene index structures. + (Areek Zillur via Robert Muir) + +* SOLR-5241: Fix SimplePostToolTest performance problem - implicit DNS lookups + (hossman) + +* SOLR-5273: Update HttpComponents to 4.2.5 and 4.2.6. (Mark Miller) + +* SOLR-5264: Move methods for getting config information from + FieldMutatingUpdateProcessorFactory to NamedList. (Shawn Heisey) + +* SOLR-5319: Remove unused and incorrect router name from Collection ZK nodes. + (Jessica Cheng via shalin) + +* SOLR-5321: Remove unnecessary code in Overseer.updateState method which tries to + use router name from message where none is ever sent. (shalin) + +* SOLR-5401: SolrResourceLoader logs a warning if a deprecated (factory) class + is used in schema or config. (Uwe Schindler) + +* SOLR-3397: Warn if master or slave replication is enabled in SolrCloud mode. (Erick + Erickson) + +================== 4.5.1 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.4 +Carrot2 3.8.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.5 + +Detailed Change List +---------------------- + +Bug Fixes +---------------------- + +* SOLR-4590: Collections API should return a nice error when not in SolrCloud mode. + (Anshum Gupta, Mark Miller) + +* SOLR-5295: The CREATESHARD collection API creates maxShardsPerNode number of + replicas if replicationFactor is not specified. (Brett Hoerner, shalin) + +* SOLR-5296: Creating a collection with implicit router adds shard ranges + to each shard. (shalin) + +* SOLR-5263: Fix CloudSolrServer URL cache update race. (Jessica Cheng, Mark Miller) + +* SOLR-5297: Admin UI - Threads Screen missing Icon (steffkes) + +* SOLR-5301: DELETEALIAS command prints CREATEALIAS in logs (janhoy) + +* SOLR-5255: Remove unnecessary call to fetch and watch live nodes in ZkStateReader + cluster watcher. (Jessica Cheng via shalin) + +* SOLR-5305: Admin UI - Reloading System-Information on Dashboard does not work + anymore (steffkes) + +* SOLR-5314: Shard split action should use soft commits instead of hard commits + to make sub shard data visible. (Kalle Aaltonen, shalin) + +* SOLR-5327: SOLR-4915, "The root cause should be returned to the user when a SolrCore create + call fails", was reverted. (Mark Miller) + +* SOLR-5317: SolrCore persistence bugs if defining SolrCores in solr.xml. + (Mark Miller, Yago Riveiro) + + * SOLR-5306: Extra collection creation parameters like collection.configName are + not being respected. (Mark Miller, Liang Tianyu, Nathan Neulinger) + +* SOLR-5325: ZooKeeper connection loss can cause the Overseer to stop processing + commands. (Christine Poerschke, Mark Miller, Jessica Cheng) + +* SOLR-4327: HttpSolrServer can leak connections on errors. (Karl Wright, Mark Miller) + +* SOLR-5349: CloudSolrServer - ZK timeout arguments passed to ZkStateReader are flipped. + (Ricardo Merizalde via shalin) + +* SOLR-5330: facet.method=fcs on single values fields could sometimes result + in incorrect facet labels. (Michael Froh, yonik) + + +Other Changes +---------------------- + +* SOLR-5323: Disable ClusteringComponent by default in collection1 example. + The solr.clustering.enabled system property needs to be set to 'true' + to enable the clustering contrib (reverts SOLR-4708). (Dawid Weiss) + +================== 4.5.0 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.4 +Carrot2 3.8.0 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.5 + +Upgrading from Solr 4.4.0 +---------------------- + +* XML configuration parsing is now more strict about situations where a single + setting is allowed but multiple values are found. In the past, one value + would be chosen arbitrarily and silently. Starting with 4.5, configuration + parsing will fail with an error in situations like this. If you see error + messages such as "solrconfig.xml contains more than one value for config path: + XXXXX" or "Found Z configuration sections when at most 1 is allowed matching + expression: XXXXX" check your solrconfig.xml file for multiple occurrences of + XXXXX and delete the ones that you do not wish to use. See SOLR-4953 & + SOLR-5108 for more details. + +* In the past, schema.xml parsing would silently ignore "default" or "required" + options specified on <dynamicField/> declarations. Beginning with 4.5, attempting + to do configured these on a dynamic field will cause an init error. If you + encounter one of these errors when upgrading an existing schema.xml, you can + safely remove these attributes, regardless of their value, from your config and + Solr will continue to behave exactly as it did in previous versions. See + SOLR-5227 for more details. + +* The UniqFieldsUpdateProcessorFactory has been improved to support all of the + FieldMutatingUpdateProcessorFactory selector options. The <lst named="fields"> + init param option is now deprecated and should be replaced with the more standard + <arr name="fieldName">. See SOLR-4249 for more details. + +* UpdateRequestExt has been removed as part of SOLR-4816. You should use UpdateRequest + instead. + +* CloudSolrServer can now use multiple threads to add documents by default. This is a + small change in runtime semantics when using the bulk add method - you will still + end up with the same exception on a failure, but some documents beyond the one that + failed may have made it in. To get the old, single threaded behavior, set parallel updates + to false on the CloudSolrServer instance. + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-5219: Rewritten selection of the default search and document clustering + algorithms. (Dawid Weiss) + +* SOLR-5202: Support easier overrides of Carrot2 clustering attributes via + XML data sets exported from the Workbench. (Dawid Weiss) + +* SOLR-5126: Update Carrot2 clustering to version 3.8.0, update Morfologik + to version 1.7.1 (Dawid Weiss) + +* SOLR-2345: Enhanced geodist() to work with an RPT field, provided that the + field is referenced via 'sfield' and the query point is constant. + (David Smiley) + +* SOLR-5082: The encoding of URL-encoded query parameters can be changed with + the "ie" (input encoding) parameter, e.g. "select?q=m%FCller&ie=ISO-8859-1". + The default is UTF-8. To change the encoding of POSTed content, use the + "Content-Type" HTTP header. (Uwe Schindler, Shawn Heisey) + +* SOLR-4221: Custom sharding (Noble Paul) + +* SOLR-4808: Persist and use router,replicationFactor and maxShardsPerNode at Collection + and Shard level (Noble Paul, Shalin Mangar) + +* SOLR-5006: CREATESHARD command for 'implicit' shards (Noble Paul) + +* SOLR-5017: Allow sharding based on the value of a field (Noble Paul) + +* SOLR-4222: create custom sharded collection via collections API (Noble Paul) + +* SOLR-4718: Allow solr.xml to be stored in ZooKeeper. (Mark Miller, Erick Erickson) + +* SOLR-5156: Enhance ZkCLI to allow uploading of arbitrary files to ZK. (Erick Erickson) + +* SOLR-5165: Single-valued docValues fields no longer require a default value. + Additionally they work with sortMissingFirst, sortMissingLast, facet.missing, + exists() in function queries, etc. (Robert Muir) + +* SOLR-5182: Add NoOpRegenerator, a regenerator for custom per-segment caches + where items are preserved across commits. (Robert Muir) + +* SOLR-4249: UniqFieldsUpdateProcessorFactory now extends + FieldMutatingUpdateProcessorFactory and supports all of its selector options. Use + of the "fields" init param is now deprecated in favor of "fieldName" (hossman) + +* SOLR-2548: Allow multiple threads to be specified for faceting. When threading, one + can specify facet.threads to parallelize loading the uninverted fields. In at least + one extreme case this reduced warmup time from 20 seconds to 3 seconds. (Janne Majaranta, + Gun Akkor via Erick Erickson, David Smiley) + +* SOLR-4816: CloudSolrServer can now route updates locally and no longer relies on inter-node + update forwarding. (Joel Bernstein, Shikhar Bhushan, Stephen Riesenberg, Mark Miller) + +* SOLR-3249: Allow CloudSolrServer and SolrCmdDistributor to use JavaBin. (Mark Miller) + +Bug Fixes +---------------------- + +* SOLR-3633: web UI reports an error if CoreAdminHandler says there are no + SolrCores (steffkes) + +* SOLR-4489: SpellCheckComponent can throw StringIndexOutOfBoundsException + when generating collations involving multiple word-break corrections. + (James Dyer) + +* SOLR-5087 - CoreAdminHandler.handleMergeAction generating NullPointerException + (Patrick Hunt via Erick Erickson) + +* SOLR-5107: Fixed NPE when using numTerms=0 in LukeRequestHandler + (Ahmet Arslan, hossman) + +* SOLR-4679, SOLR-4908, SOLR-5124: Text extracted from HTML or PDF files + using Solr Cell was missing ignorable whitespace, which is inserted by + TIKA for convenience to support plain text extraction without using the + HTML elements. This bug resulted in glued words. (hossman, Uwe Schindler) + +* SOLR-5121: zkcli usage help for makepath doesn't match actual command. + (Daniel Collins via Mark Miller) + +* SOLR-5119: Managed schema problems after adding fields via Schema Rest API. + (Nils Kübler, Steve Rowe) + +* SOLR-5133: HdfsUpdateLog can fail to close a FileSystem instance if init + is called more than once. (Mark Miller) + +* SOLR-5135: Harden Collection API deletion of /collections/$collection + ZooKeeper node. (Mark Miller) + +* SOLR-4764: When using NRT, just init the first reader from IndexWriter. + (Robert Muir, Mark Miller) + +* SOLR-5122: Fixed bug in spellcheck.collateMaxCollectDocs. Eliminates risk + of divide by zero, and makes estimated hit counts meaningful in non-optimized + indexes. (hossman) + +* SOLR-3936: Fixed QueryElevationComponent sorting when used with Grouping + (Michael Garski via hossman) + +* SOLR-5171: SOLR Admin gui works in IE9, breaks in IE10. (Joseph L Howard via + steffkes) + +* SOLR-5174: Admin UI - Query View doesn't highlight (json) Result if it + contains HTML Tags (steffkes) + +* SOLR-4817 Solr should not fall back to the back compat built in solr.xml in SolrCloud + mode (Erick Erickson) + +* SOLR-5112: Show full message in Admin UI Logging View (Matthew Keeney via + steffkes) + +* SOLR-5190: SolrEntityProcessor substitutes variables only once in child entities + (Harsh Chawla, shalin) + +* SOLR-3852: Fixed ZookeeperInfoServlet so that the SolrCloud Admin UI pages will + work even if ZK contains nodes with data which are not utf8 text. (hossman) + +* SOLR-5206: Fixed OpenExchangeRatesOrgProvider to use refreshInterval correctly + (Catalin, hossman) + +* SOLR-5215: Fix possibility of deadlock in ZooKeeper ConnectionManager. + (Mark Miller, Ricardo Merizalde) + +* SOLR-4909: Use DirectoryReader.openIfChanged in non-NRT mode. + (Michael Garski via Robert Muir) + +* SOLR-5227: Correctly fail schema initialization if a dynamicField is configured to + be required, or have a default value. (hossman) + +* SOLR-5231: Fixed a bug with the behavior of BoolField that caused documents w/o + a value for the field to act as if the value were true in functions if no other + documents in the same index segment had a value of true. + (Robert Muir, hossman, yonik) + +* SOLR-5233: The "deleteshard" collections API doesn't wait for cluster state to update, + can fail if some nodes of the deleted shard were down and had incorrect logging. + (Christine Poerschke, shalin) + +* SOLR-5150: HdfsIndexInput may not fully read requested bytes. (Mark Miller, Patrick Hunt) + +* SOLR-5240: All solr cores will now be loaded in parallel (as opposed to a fixed number) + in zookeeper mode to avoid deadlocks due to replicas waiting for other replicas + to come up. (yonik) + +* SOLR-5243: Killing a shard in one collection can result in leader election in a different + collection if they share the same coreNodeName. (yonik, Mark Miller) + +* SOLR-5281: IndexSchema log message was printing '[null]' instead of + '[<core name>]' (Jun Ohtani via Steve Rowe) + +* SOLR-5279: Implicit properties don't seem to exist on core RELOAD + (elyograg, hossman, Steve Rowe) + +* SOLR-5291: Solrj does not propagate the root cause to the user for many errors. + (Mark Miller) + +Optimizations +---------------------- + +* SOLR-5044: Admin UI - Note on Core-Admin about directories while creating + core (steffkes) + +* SOLR-5134: Have HdfsIndexOutput extend BufferedIndexOutput. + (Mark Miller, Uwe Schindler) + + * SOLR-5057: QueryResultCache should not related with the order of fq's list (Feihong Huang via Erick Erickson) + +* SOLR-4816: CloudSolrServer now uses multiple threads to send updates by default. + (Joel Bernstein via Mark Miller) + +* SOLR-3530: Better error messages / Content-Type validation in SolrJ. (Mark Miller, hossman) + +Other Changes +---------------------- + +* SOLR-4708: Enable ClusteringComponent by default in collection1 example. + The solr.clustering.enabled system property is set to 'true' by default. + (ehatcher, Dawid Weiss) + +* SOLR-4914, SOLR-5162: Factor out core list persistence and discovery into a + new CoresLocator interface. (Alan Woodward, Shawn Heisey) + +* SOLR-5056: Improve type safety of ConfigSolr class. (Alan Woodward) + +* SOLR-4951: Better randomization of MergePolicy in Solr tests (hossman) + +* SOLR-4953, SOLR-5108: Make XML Configuration parsing fail if an xpath matches + multiple nodes when only a single value or plugin instance is expected. + (hossman) + +* The routing parameter "shard.keys" is deprecated as part of SOLR-5017 .The new parameter name is '_route_' . + The old parameter should continue to work for another release (Noble Paul) + +* SOLR-5173: Solr-core's Maven configuration includes test-only Hadoop + dependencies as indirect compile-time dependencies. + (Chris Collins, Steve Rowe) + +================== 4.4.0 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.4 +Carrot2 3.6.2 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.5 + +Upgrading from Solr 4.3.0 +---------------------- + +* TieredMergePolicy and the various subtypes of LogMergePolicy no longer have + an explicit "setUseCompoundFile" method. Instead the behavior of new + segments is determined by the IndexWriter configuration, and the MergePolicy + is only consulted to determine if merge segments should use the compound + file format (based on the value of "setNoCFSRatio"). If you have explicitly + configured one of these classes using <mergePolicy> and include an init arg + like this... + <bool name="useCompoundFile">true</bool> + ...this will now be treated as if you specified... + <useCompoundFile>true</useCompoundFile> + ...directly on the <indexConfig> (overriding any value already set using that + syntax) and a warning will be logged to updated your configuration. Users + with an explicitly declared <mergePolicy> are encouraged to review the + current javadocs for their MergePolicy subclass and review their configured + options carefully. See SOLR-4941, SOLR-4934 and LUCENE-5038 for more + information. + +* SOLR-4778: The signature of LogWatcher.registerListener has changed, from + (ListenerConfig, CoreContainer) to (ListenerConfig). Users implementing their + own LogWatcher classes will need to change their code accordingly. + +* LUCENE-5063: ByteField and ShortField have been deprecated and will be removed + in 5.0. If you are still using these field types, you should migrate your + fields to TrieIntField. + + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-3251: Dynamically add fields to schema. (Steve Rowe, Robert Muir, yonik) + +* SOLR-4761, SOLR-4976: Add option to plugin a merged segment warmer into solrconfig.xml. + Info about segments warmed in the background is available via infostream. + (Mark Miller, Ryan Ernst, Mike McCandless, Robert Muir) + +* SOLR-3240: Add "spellcheck.collateMaxCollectDocs" option so that when testing + potential Collations against the index, SpellCheckComponent will only collect + n documents, thereby estimating the hit-count. This is a performance optimization + in cases where exact hit-counts are unnecessary. Also, when "collateExtendedResults" + is false, this optimization is always made (James Dyer). + +* SOLR-4785: New MaxScoreQParserPlugin returning max() instead of sum() of terms (janhoy) + +* SOLR-4234: Add support for binary files in ZooKeeper. (Eric Pugh via Mark Miller) + +* SOLR-4048: Add findRecursive method to NamedList. (Shawn Heisey) + +* SOLR-4228: SolrJ's SolrPing object has new methods for ping, enable, and + disable. (Shawn Heisey, hossman, Steve Rowe) + +* SOLR-4893: Extend FieldMutatingUpdateProcessor.ConfigurableFieldNameSelector + to enable checking whether a field matches any schema field. To select field + names that don't match any fields or dynamic fields in the schema, add + <bool name="fieldNameMatchesSchemaField">false</bool> to an update + processor's configuration in solrconfig.xml. (Steve Rowe, hossman) + +* SOLR-4921: Admin UI now supports adding documents to Solr (gsingers, steffkes) + +* SOLR-4916: Add support to write and read Solr index files and transaction log + files to and from HDFS. (phunt, Mark Miller, Gregory Chanan) + +* SOLR-4892: Add FieldMutatingUpdateProcessorFactory subclasses + Parse{Date,Integer,Long,Float,Double,Boolean}UpdateProcessorFactory. These + factories have a default selector that matches all fields that either don’t + match any schema field, or are in the schema with the corresponding + typeClass. If they see a value that is not a CharSequence, or can't parse + the value, they leave it as is. For multi-valued fields, these processors + will not convert any values unless all are first successfully parsed, or + already are instances of the target class. Ordering the processors, e.g. + [Boolean, Long, Double, Date] will allow e.g. values ["2", "5", "8.6"] to + be left alone by the Boolean and Long processors, but then converted by the + Double processor. (Steve Rowe, hossman) + +* SOLR-4972: Add PUT command to ZkCli tool. (Roman Shaposhnik via Mark Miller) + +* SOLR-4973: Adding getter method for defaultCollection on CloudSolrServer. + (Furkan KAMACI via Mark Miller) + +* SOLR-4897: Add solr/example/example-schemaless/, an example config set + for schemaless mode. (Steve Rowe) + +* SOLR-4655: Add option to have Overseer assign generic node names so that + new addresses can host shards without naming confusion. (Mark Miller, Anshum Gupta) + +* SOLR-4977: Add option to send IndexWriter's infostream to the logging system. + (Ryan Ernst via Robert Muir) + +* SOLR-4693: A "deleteshard" collections API that unloads all replicas of a given + shard and then removes it from the cluster state. It will remove only those shards + which are INACTIVE or have no range (created for custom sharding). + (Anshum Gupta, shalin) + +* SOLR-5003: CSV Update Handler supports optionally adding the line number/row id to + a document (gsingers) + +* SOLR-5010: Add support for creating copy fields to the Schema REST API (gsingers) + +* SOLR-4991: Register QParserPlugins as SolrInfoMBeans (ehatcher) + +* SOLR-4943: Add a new system wide info admin handler that exposes the system info + that could previously only be retrieved using a SolrCore. (Mark Miller) + +* SOLR-3076: Block joins. Documents and their sub-documents must be indexed + as a block. + {!parent which=<allParents>}<someChildren> takes in a query that matches child + documents and results in matches on their parents. + {!child of=<allParents>}<someParents> takes in a query that matches some parent + documents and results in matches on their children. + (Mikhail Khludnev, Vadim Kirilchuk, Alan Woodward, Tom Burton-West, Mike McCandless, + hossman, yonik) + + +Bug Fixes +---------------------- + +* SOLR-4333: edismax parser to not double-escape colons if already escaped by + the client application (James Dyer, Robert J. van der Boon) + +* SOLR-4776: Solrj doesn't return "between" count in range facets + (Philip K. Warren via shalin) + +* SOLR-4616: HitRatio on caches is now exposed over JMX MBeans as a float. + (Greg Bowyer) + +* SOLR-4803: Fixed core discovery mode (ie: new style solr.xml) to treat + 'collection1' as the default core name. (hossman) + +* SOLR-4790: Throw an error if a core has the same name as another core, both old and + new style solr.xml + +* SOLR-4842: Fix facet.field local params from affecting other facet.field's. + (ehatcher, hossman) + +* SOLR-4814: If a SolrCore cannot be created it should remove any information it + published about itself from ZooKeeper. (Mark Miller) + +* SOLR-4863: Removed non-existent attribute sourceId from dynamic JMX stats + to fix AttributeNotFoundException (suganuma, hossman via shalin) + +* SOLR-4891: JsonLoader should preserve field value types from the JSON content stream. + (Steve Rowe) + +* SOLR-4805: SolrCore#reload should not call preRegister and publish a DOWN state to + ZooKeeper. (Mark Miller, Jared Rodriguez) + +* SOLR-4899: When reconnecting after ZooKeeper expiration, we need to be willing to wait + forever, not just for 30 seconds. (Mark Miller) + +* SOLR-4920: JdbcDataSource incorrectly suppresses exceptions when retrieving a connection from + a JNDI context and falls back to trying to use DriverManager to obtain a connection. Additionally, + if a SQLException is thrown while initializing a connection, such as in setAutoCommit(), the + connection will not be closed. (Chris Eldredge via shalin) + +* SOLR-4915: The root cause should be returned to the user when a SolrCore create call fails. + (Mark Miller) + +* SOLR-4925 : Collection create throws NPE when 'numShards' param is missing (Noble Paul) + +* SOLR-4910: persisting solr.xml is broken. More stringent testing of persistence fixed + up a number of issues and several bugs with persistence. Among them are + - don't persisting implicit properties + - should persist zkHost in the <solr> tag (user's list) + - reloading a core that has transient="true" returned an error. reload should load + a transient core if it's not yet loaded. + - No longer persisting loadOnStartup or transient core properties if they were not + specified in the original solr.xml + - Testing flushed out the fact that you couldn't swap a core marked transient=true + loadOnStartup=false because it hadn't been loaded yet. + - SOLR-4862, CREATE fails to persist schema, config, and dataDir + - SOLR-4363, not persisting coreLoadThreads in <solr> tag + - SOLR-3900, logWatcher properties not persisted + - SOLR-4850, cores defined as loadOnStartup=true, transient=false can't be searched + (Erick Erickson) + +* SOLR-4923: Commits to non leaders as part of a request that also contain updates + can execute out of order. (hossman, Ricardo Merizalde, Mark Miller) + +* SOLR-4932: persisting solr.xml saves some parameters it shouldn't when they weren't + defined in the original. Benign since the default values are saved, but still incorrect. + (Erick Erickson, thanks Shawn Heisey for helping test!) + +* SOLR-4934, SOLR-4941: Fix handling of <mergePolicy> init arg + "useCompoundFile" needed after changes in LUCENE-5038 (hossman) + +* SOLR-4456: Admin UI: Displays dashboard even if Solr is down (steffkes) + +* SOLR-4949: UI Analysis page dropping characters from input box (steffkes) + +* SOLR-4960: Fix race conditions in shutdown of CoreContainer + and getCore that could cause a request to attempt to use a core that + has shut down. (yonik) + +* SOLR-4926: Fixed rare replication bug that normally only manifested when + using compound file format. (yonik, Mark Miller) + +* SOLR-4974: Outgrowth of SOLR-4960 that includes transient cores and pending cores + (Erick Erickson) + +* SOLR-3369: shards.tolerant=true is broken for group queries + (Russell Black, Martijn van Groningen, Jabouille jean Charles, Ryan McKinley via shalin) + +* SOLR-4452: Hunspell stemmer should not merge duplicate dictionary entries (janhoy) + +* SOLR-5000: ManagedIndexSchema doesn't persist uniqueKey tag after calling addFields + method. (Jun Ohtani, Steve Rowe) + +* SOLR-4982: Creating a core while referencing system properties looks like it loses files + Actually, instanceDir, config, dataDir and schema are not dereferenced properly + when creating cores that reference sys vars (e.g. &dataDir=${dir}). In the dataDir + case in particular this leads to the index being put in a directory literally named + ${dir} but on restart the sysvar will be properly dereferenced. + +* SOLR-4788: Multiple Entities DIH delta import: dataimporter.[entityName].last_index_time + is empty. (chakming wong, James Dyer via shalin) + +* SOLR-4978: Time is stripped from datetime column when imported into Solr date field + if convertType=true. (Bill Au, shalin) + +* SOLR-5019: spurious ConcurrentModificationException when spell check component + was in use with filters. (yonik) + +* SOLR-5018: The Overseer should avoid publishing the state for collections that do not + exist under the /collections zk node. (Mark Miller) + +* SOLR-5028,SOLR-5029: ShardHandlerFactory was not being created properly when + using new-style solr.xml, and was not being persisted properly when using + old-style. (Tomás Fernández Löbbe, Ryan Ernst, Alan Woodward) + +* SOLR-4997: The splitshard api doesn't call commit on new sub shards before + switching shard states. Multiple bugs related to sub shard recovery and + replication are also fixed. (shalin) + +* SOLR-5034: A facet.query that parses or analyzes down to a null Query would + throw a NPE. Fixed. (David Smiley) + +* SOLR-5039: Admin/Schema Browser displays -1 for term counts for multiValued fields. + +* SOLR-5037: The CSV loader now accepts field names that are not in the schema. + (gsingers, ehatcher, Steve Rowe) + +* SOLR-4791: solr.xml sharedLib does not work in 4.3.0 (Ryan Ernst, Jan Høydahl via + Erick Erickson) + +Optimizations +---------------------- + +* SOLR-4923: Commit to all nodes in a collection in parallel rather than locally and + then to all other nodes. (hossman, Ricardo Merizalde, Mark Miller) + +* SOLR-3838: Admin UI - Multiple filter queries are not supported in Query UI (steffkes) + +* SOLR-4719 : Admin UI - Default to wt=json on Query-Screen (steffkes) + +* SOLR-4611: Admin UI - Analysis-Urls with empty parameters create empty result table + (steffkes) + +* SOLR-4955: Admin UI - Show address bar on top for Schema + Config (steffkes) + +* SOLR-4412: New parameter langid.lcmap to map detected language code to be placed + in "language" field (janhoy) + +* SOLR-4815: Admin-UI - DIH: Let "commit" be checked by default (steffkes) + +* SOLR-5002: optimize numDocs(Query,DocSet) when filterCache is null (Robert Muir) + +* SOLR-5012: optimize search with filter when filterCache is null (Robert Muir) + +Other Changes +---------------------- + +* SOLR-4737: Update Guava to 14.0.1 (Mark Miller) + +* SOLR-2079: Add option to pass HttpServletRequest in the SolrQueryRequest context map. + (Tomás Fernández Löbbe via Robert Muir) + +* SOLR-4738: Update Jetty to 8.1.10.v20130312 (Mark Miller, Robert Muir) + +* SOLR-4749: Clean up and refactor CoreContainer code around solr.xml and SolrCore + management. (Mark Miller) + +* SOLR-4547: Move logging of filenames on commit from INFO to DEBUG. + (Shawn Heisey, hossman) + +* SOLR-4757: Change the example to use the new solr.xml format and core + discovery by directory structure. (Mark Miller) + +* SOLR-4759: Velocity (/browse) template cosmetic cleanup. + (Mark Bennett, ehatcher) + +* SOLR-4778: LogWatcher init code moved out of CoreContainer (Alan Woodward) + +* SOLR-4784: Make class LuceneQParser public (janhoy) + +* SOLR-4448: Allow the solr internal load balancer to be more easily pluggable. + (Philip Hoy via Robert Muir) + +* SOLR-4224: Refactor JavaBinCodec input stream definition to enhance reuse. + (phunt via Mark Miller) + +* SOLR-4931: SolrDeletionPolicy onInit and onCommit methods changed to override + exact signatures (with generics) from IndexDeletionPolicy (shalin) + +* SOLR-4942: test improvements to randomize use of compound files (hossman) + +* SOLR-4966: CSS, JS and other files in webapp without license (uschindler, + steffkes) + +* SOLR-4986: Upgrade to Tika 1.4 (Markus Jelsma via janhoy) + +* SOLR-4948, SOLR-5009: Tidied up CoreContainer construction logic. + (Alan Woodward, Uwe Schindler, Steve Rowe) + +* LUCENE-5107: Properties files by Solr are now written in UTF-8 encoding, + Unicode is no longer escaped. Reading of legacy properties files with + \u escapes is still possible. (Uwe Schindler, Robert Muir) + +================== 4.3.1 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.3 +Carrot2 3.6.2 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.5 + +Detailed Change List +---------------------- + +Bug Fixes +---------------------- + +* SOLR-4795: Sub shard leader should not accept any updates from parent after + it goes active (shalin) + +* SOLR-4798: shard splitting does not respect the router for the collection + when executing the index split. One effect of this is that documents + may be placed in the wrong shard when the default compositeId router + is used in conjunction with IDs containing "!". (yonik) + +* SOLR-4797: Shard splitting creates sub shards which have the wrong hash + range in cluster state. This happens when numShards is not a power of two + and router is compositeId. (shalin) + +* SOLR-4806: Shard splitting does not abort if WaitForState times out (shalin) + +* SOLR-4807: The zkcli script now works with log4j. The zkcli.bat script + was broken on Windows in 4.3.0, now it works. (Shawn Heisey) + +* SOLR-4813: Fix SynonymFilterFactory to allow init parameters for + tokenizer factory used when parsing synonyms file. (Shingo Sasaki, hossman) + +* SOLR-4829: Fix transaction log leaks (a failure to clean up some old logs) + on a shard leader, or when unexpected exceptions are thrown during log + recovery. (Steven Bower, Mark Miller, yonik) + +* SOLR-4751: Fix replication problem of files in sub directory of conf directory. + (Minoru Osuka via Koji) + +* SOLR-4741: Deleting a collection should set DELETE_DATA_DIR to true. + (Mark Miller) + +* SOLR-4752: There are some minor bugs in the Collections API parameter + validation. (Mark Miller) + +* SOLR-4563: RSS DIH-example not working (janhoy) + +* SOLR-4796: zkcli.sh should honor JAVA_HOME (Roman Shaposhnik via Mark Miller) + +* SOLR-4734: Leader election fails with an NPE if there is no UpdateLog. + (Mark Miller, Alexander Eibner) + +* SOLR-4868: Setting the log level for the log4j root category results in + adding a new category, the empty string. (Shawn Heisey) + +* SOLR-4855: DistributedUpdateProcessor doesn't check for peer sync requests (shalin) + +* SOLR-4867: Admin UI - setting loglevel on root throws RangeError (steffkes) + +* SOLR-4870: RecentUpdates.update() does not increment numUpdates loop counter + (Alexey Kudinov via shalin) + +* SOLR-4877, LUCENE-5023: Removed SolrIndexSearcher#getDocSetNC()'s special + case for handling TermQuery to prevent NullPointerException if reader does + not have fields. (Bao Yang Yang, Uwe Schindler) + +* SOLR-4881: Fix DocumentAnalysisRequestHandler to correctly use + EmptyEntityResolver to prevent loading of external entities like + UpdateRequestHandler does. (Hossman, Uwe Schindler) + +* SOLR-4858: SolrCore reloading was broken when the UpdateLog + was enabled. (Hossman, Anshum Gupta, Alexey Serba, Mark Miller, yonik) + +* SOLR-4853: Fixed SolrJettyTestBase so it may be reused by end users + (hossman) + +* SOLR-4744: Update failure on sub shard is not propagated to clients by parent + shard (Anshum Gupta, yonik, shalin) + +Other Changes +---------------------- + +* SOLR-4760: Include core name in logs when loading schema. + (Shawn Heisey) + +================== 4.3.0 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.3 +Carrot2 3.6.2 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.5 + +Upgrading from Solr 4.2.0 +---------------------- + +* In the schema REST API, the output path for copyFields and dynamicFields + has been changed from all lowercase "copyfields" and "dynamicfields" to + camelCase "copyFields" and "dynamicFields", respectively, to align with all + other schema REST API outputs, which use camelCase. The URL format remains + the same: all resource names are lowercase. See SOLR-4623 for details. + +* Slf4j/logging jars are no longer included in the Solr webapp. All logging + jars are now in example/lib/ext. Changing logging impls is now as easy as + updating the jars in this folder with those necessary for the logging impl + you would like. If you are using another webapp container, these jars will + need to go in the corresponding location for that container. + In conjunction, the dist-excl-slf4j and dist-war-excl-slf4 build targets + have been removed since they are redundant. See the Slf4j documentation, + SOLR-3706, and SOLR-4651 for more details. + +* The hardcoded SolrCloud defaults for 'hostContext="solr"' and + 'hostPort="8983"' have been deprecated and will be removed in Solr 5.0. + Existing solr.xml files that do not have these options explicitly specified + should be updated accordingly. See SOLR-4622 for more details. + + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-4648 PreAnalyzedUpdateProcessorFactory allows using the functionality + of PreAnalyzedField with other field types. See javadoc for details and + examples. (Andrzej Bialecki) + +* SOLR-4623: Provide REST API read access to all elements of the live schema. + Add a REST API request to return the entire live schema, in JSON, XML, and + schema.xml formats. Move REST API methods from package org.apache.solr.rest + to org.apache.solr.rest.schema, and rename base functionality REST API + classes to remove the current schema focus, to prepare for other non-schema + REST APIs. Change output path for copyFields and dynamicFields from + "copyfields" and "dynamicfields" (all lowercase) to "copyFields" and + "dynamicFields", respectively, to align with all other REST API outputs, which + use camelCase. + (Steve Rowe) + +* SOLR-4658: In preparation for REST API requests that can modify the schema, + a "managed schema" is introduced. + Add '<schemaFactory class="ManagedSchemaFactory" mutable="true"/>' to solrconfig.xml + in order to use it, and to enable schema modifications via REST API requests. + (Steve Rowe, Robert Muir) + +* SOLR-4656: Added two new highlight parameters, hl.maxMultiValuedToMatch and + hl.maxMultiValuedToExamine. maxMultiValuedToMatch stops looking for snippets after + finding the specified number of matches, no matter how far into the multivalued field + you've gone. maxMultiValuedToExamine stops looking for matches after the specified + number of multiValued entries have been examined. If both are specified, the limit + hit first stops the loop. Also this patch cuts down on the copying of the document + entries during highlighting. These optimizations are probably unnoticeable unless + there are a large number of entries in the multiValued field. Conspicuously, this will + prevent the "best" match from being found if it appears later in the MV list than the + cutoff specified by either of these params. (Erick Erickson) + +* SOLR-4675: Improve PostingsSolrHighlighter to support per-field/query-time overrides + and add additional configuration parameters. See the javadocs for more details and + examples. (Robert Muir) + +* SOLR-3755: A new collections api to add additional shards dynamically by splitting + existing shards. (yonik, Anshum Gupta, shalin) + +* SOLR-4530: DIH: Provide configuration to use Tika's IdentityHtmlMapper + (Alexandre Rafalovitch via shalin) + +* SOLR-4662: Discover SolrCores by directory structure rather than defining them + in solr.xml. Also, change the format of solr.xml to be closer to that of solrconfig.xml. + This version of Solr will ship the example in the old style, but you can manually + try the new style. Solr 4.4 will ship with the new style, and Solr 5.0 will remove + support for the old style. (Erick Erickson, Mark Miller) + Additional Work: + - SOLR-4347: Ensure that newly-created cores via Admin handler are persisted in solr.xml + (Erick Erickson) + - SOLR-1905: Cores created by the admin request handler should be persisted to solr.xml. + Also fixed a problem whereby properties like solr.solr.datadir would be persisted + to solr.xml. Also, cores that didn't happen to be loaded were not persisted. + (Erick Erickson) + +* SOLR-4717/SOLR-1351: SimpleFacets now work with localParams allowing faceting on the + same field multiple ways (ryan, Uri Boness) + +* SOLR-4671: CSVResponseWriter now supports pseudo fields. (ryan, nihed mbarek) + +* SOLR-4358: HttpSolrServer sends the stream name and exposes 'useMultiPartPost' + (Karl Wright via ryan) + + +Bug Fixes +---------------------- + +* SOLR-4543: setting shardHandlerFactory in solr.xml/solr.properties does not work. + (Ryan Ernst, Robert Muir via Erick Erickson) + +* SOLR-4634: Fix scripting engine tests to work with Java 8's "Nashorn" Javascript + implementation. (Uwe Schindler) + +* SOLR-4636: If opening a reader fails for some reason when opening a SolrIndexSearcher, + a Directory can be left unreleased. (Mark Miller) + +* SOLR-4405: Admin UI - admin-extra files are not rendered into the core-menu (steffkes) + +* SOLR-3956: Fixed group.facet=true to work with negative facet.limit + (Chris van der Merwe, hossman) + +* SOLR-4650: copyField doesn't work with source globs that don't match any + explicit or dynamic fields. This regression was introduced in Solr 4.2. + (Daniel Collins, Steve Rowe) + +* SOLR-4641: Schema now throws exception on illegal field parameters. (Robert Muir) + +* SOLR-3758: Fixed SpellCheckComponent to work consistently with distributed grouping + (James Dyer) + +* SOLR-4652: Fix broken behavior with shared libraries in resource loader for + solr.xml plugins. (Ryan Ernst, Robert Muir, Uwe Schindler) + +* SOLR-4664: ZkStateReader should update aliases on construction. + (Mark Miller, Elodie Sannier) + +* SOLR-4682: CoreAdminRequest.mergeIndexes can not merge multiple cores or indexDirs. + (Jason.D.Cao via shalin) + +* SOLR-4581: When faceting on numeric fields in Solr 4.2, negative values (constraints) + were sorted incorrectly. (Alexander Buhr, shalin, yonik) + +* SOLR-4699: The System admin handler should not assume a file system based data directory + location. (Mark Miller) + +* SOLR-4695: Fix core admin SPLIT action to be useful with non-cloud setups (shalin) + +* SOLR-4680: Correct example spellcheck configuration's queryAnalyzerFieldType and + use "text" field instead of narrower "name" field (ehatcher, Mark Bennett) + +* SOLR-4702: Fix example /browse "Did you mean?" suggestion feature. (ehatcher, Mark Bennett) + +* SOLR-4710: You cannot delete a collection fully from ZooKeeper unless all nodes are up and + functioning correctly. (Mark Miller) + +* SOLR-4487: SolrExceptions thrown by HttpSolrServer will now contain the + proper HTTP status code returned by the remote server, even if that status + code is not something Solr itself returned -- eg: from the Servlet Container, + or an intermediate HTTP Proxy (hossman) + +* SOLR-4661: Admin UI Replication details now correctly displays the current + replicable generation/version of the master. (hossman) + +* SOLR-4716,SOLR-4584: SolrCloud request proxying does not work on Tomcat and + perhaps other non Jetty containers. (Po Rui, Yago Riveiro via Mark Miller) + +* SOLR-4746: Distributed grouping used a NamedList instead of a SimpleOrderedMap + for the top level group commands, causing output formatting differences + compared to non-distributed grouping. (yonik) + +* SOLR-4705: Fixed bug causing NPE when querying a single replica in SolrCloud + using the shards param (Raintung Li, hossman) + +* SOLR-4729: LukeRequestHandler: Using a dynamic copyField source that is + not also a dynamic field triggers error message 'undefined field: "(glob)"'. + (Adam Hahn, hossman, Steve Rowe) + +Optimizations +---------------------- + +Other Changes +---------------------- + +* SOLR-4653: Solr configuration should log inaccessible/ non-existent relative paths in lib + dir=... (Dawid Weiss) + +* SOLR-4317: SolrTestCaseJ4: Can't avoid "collection1" convention (Tricia Jenkins, via Erick Erickson) + +* SOLR-4571: SolrZkClient#setData should return Stat object. (Mark Miller) + +* SOLR-4603: CachingDirectoryFactory should use an IdentityHashMap for + byDirectoryCache. (Mark Miller) + +* SOLR-4544: Refactor HttpShardHandlerFactory so load-balancing logic can be customized. + (Ryan Ernst via Robert Muir) + +* SOLR-4607: Use noggit 0.5 release jar rather than a forked copy. (Yonik Seeley, Robert Muir) + +* SOLR-3706: Ship setup to log with log4j. (ryan, Mark Miller) + +* SOLR-4651: Remove dist-excl-slf4j build target. (Shawn Heisey) + +* SOLR-4622: The hardcoded SolrCloud defaults for 'hostContext="solr"' and + 'hostPort="8983"' have been deprecated and will be removed in Solr 5.0. + Existing solr.xml files that do not have these options explicitly specified + should be updated accordingly. (hossman) + +* SOLR-4672: Requests attempting to use SolrCores which had init failures + (that would be reported by CoreAdmin STATUS requests) now result in 500 + error responses with the details about the init failure, instead of 404 + error responses. (hossman) + +* SOLR-4730: Make the wiki link more prominent in the release documentation. + (Uri Laserson via Robert Muir) + + +================== 4.2.1 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.3 +Carrot2 3.6.2 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.5 + +Detailed Change List +---------------------- + +Bug Fixes +---------------------- + +* SOLR-4567: copyField source glob matching explicit field(s) stopped working + in Solr 4.2. (Alexandre Rafalovitch, Steve Rowe) + +* SOLR-4475: Fix various places that still assume File based paths even when + not using a file based DirectoryFactory. (Mark Miller) + +* SOLR-4551: CachingDirectoryFactory needs to create CacheEntry's with the + fullpath not path. (Mark Miller) + +* SOLR-4555: When forceNew is used with CachingDirectoryFactory#get, the old + CacheValue should give up its path as it will be used by a new Directory + instance. (Mark Miller) + +* SOLR-4578: CoreAdminHandler#handleCreateAction gets a SolrCore and does not + close it in SolrCloud mode when a core with the same name already exists. + (Mark Miller) + +* SOLR-4574: The Collections API will silently return success on an unknown + ACTION parameter. (Mark Miller) + +* SOLR-4576: Collections API validation errors should cause an exception on + clients and otherwise act as validation errors with the Core Admin API. + (Mark Miller) + +* SOLR-4577: The collections API should return responses (success or failure) + for each node it attempts to work with. (Mark Miller) + +* SOLR-4568: The lastPublished state check before becoming a leader is not + working correctly. (Mark Miller) + +* SOLR-4570: Even if an explicit shard id is used, ZkController#preRegister + should still wait to see the shard id in its current ClusterState. + (Mark Miller) + +* SOLR-4585: The Collections API validates numShards with < 0 but should use + <= 0. (Mark Miller) + +* SOLR-4592: DefaultSolrCoreState#doRecovery needs to check the CoreContainer + shutdown flag inside the recoveryLock sync block. (Mark Miller) + +* SOLR-4595: CachingDirectoryFactory#close can throw a concurrent + modification exception. (Mark Miller) + +* SOLR-4573: Accessing Admin UI files in SolrCloud mode logs warnings. + (Mark Miller, Phil John) + +* SOLR-4594: StandardDirectoryFactory#remove accesses byDirectoryCache + without a lock. (Mark Miller) + +* SOLR-4597: CachingDirectoryFactory#remove should not attempt to empty/remove + the index right away but flag for removal after close. (Mark Miller) + +* SOLR-4598: The Core Admin unload command's option 'deleteDataDir', should use + the DirectoryFactory API to remove the data dir. (Mark Miller) + +* SOLR-4599: CachingDirectoryFactory calls close(Directory) on forceNew if the + Directory has a refCnt of 0, but it should call closeDirectory(CacheValue). + (Mark Miller) + +* SOLR-4602: ZkController#unregister should cancel its election participation + before asking the Overseer to delete the SolrCore information. (Mark Miller) + +* SOLR-4601: A Collection that is only partially created and then deleted will + leave pre allocated shard information in ZooKeeper. (Mark Miller) + +* SOLR-4604: UpdateLog#init is over called on SolrCore#reload. (Mark Miller) + +* SOLR-4605: Rollback does not work correctly. (Mark S, Mark Miller) + +* SOLR-4609: The Collections API should only send the reload command to ACTIVE + cores. (Mark Miller) + +* SOLR-4297: Atomic update request containing null=true sets all subsequent + fields to null (Ben Pennell, Rob, shalin) + +* SOLR-4371: Admin UI - Analysis Screen shows empty result (steffkes) + +* SOLR-4318: NPE encountered with querying with wildcards on a field that uses + the DefaultAnalyzer (i.e. no analysis chain defined). (Erick Erickson) + +* SOLR-4361: DataImportHandler would throw UnsupportedOperationException if + handler-level parameters were specified containing periods in the name + (James Dyer) + +* SOLR-4538: Date Math expressions were being truncated to 32 characters + when used in field:value queries in the lucene QParser. (hossman, yonik) + +* SOLR-4617: SolrCore#reload needs to pass the deletion policy to the next + SolrCore through its constructor rather than setting a field after. + (Mark Miller) + +* SOLR-4589: Fixed CPU spikes and poor performance in lazy field loading + of multivalued fields. (hossman) + +* SOLR-4608: Update Log replay and PeerSync replay should use the default + processor chain to update the index. (Ludovic Boutros, yonik) + +* SOLR-4625: The solr (lucene syntax) query parser lost top-level boost + values and top-level phrase slops on queries produced by nested + sub-parsers. (yonik) + +* SOLR-4624: CachingDirectoryFactory does not need to support forceNew any + longer and it appears to be causing a missing close directory bug. forceNew + is no longer respected and will be removed in 4.3. (Mark Miller) + +* SOLR-3819: Grouped faceting (group.facet=true) did not respect filter + exclusions. (Petter Remen, yonik) + +* SOLR-4637: Replication can sometimes wait until shutdown or core unload until + removing some tmp directories. (Mark Miller) + +* SOLR-4638: DefaultSolrCoreState#getIndexWriter(null) is a way to avoid + creating the IndexWriter earlier than necessary, but it's not + implemented quite right. (Mark Miller) + +* SOLR-4640: CachingDirectoryFactory can fail to close directories in some race + conditions. (Mark Miller) + +* SOLR-4642: QueryResultKey is not calculating the correct hashCode for filters. + (Joel Bernstein via Mark Miller) + +Optimizations +---------------------- + +* SOLR-4569: waitForReplicasToComeUp should bail right away if it doesn't see the + expected slice in the clusterstate rather than waiting. (Mark Miller) + +* SOLR-4311: Admin UI - Optimize Caching Behaviour (steffkes) + +Other Changes +---------------------- + +* SOLR-4537: Clean up schema information REST API. (Steve Rowe) + +* SOLR-4596: DistributedQueue should ensure its full path exists in the constructor. + (Mark Miller) + +================== 4.2.0 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.3 +Carrot2 3.6.2 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.5 + +Upgrading from Solr 4.1.0 +---------------------- + +(No upgrade instructions yet) + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-4043: Add ability to get success/failure responses from Collections API. + (Raintung Li, Mark Miller) + +* SOLR-2827: RegexpBoost Update Processor (janhoy) + +* SOLR-4370: Allow configuring commitWithin to do hard commits. + (Mark Miller, Senthuran Sivananthan) + +* SOLR-4451: SolrJ, and SolrCloud internals, now use SystemDefaultHttpClient + under the covers -- allowing many HTTP connection related properties to be + controlled via 'standard' java system properties. (hossman) + +* SOLR-3855, SOLR-4490: Doc values support. (Adrien Grand, Robert Muir) + +* SOLR-4417: Reopen the IndexWriter on SolrCore reload. (Mark Miller) + +* SOLR-4477: Add support for queries (match-only) against docvalues fields. + (Robert Muir) + +* SOLR-4488: Return slave replication details for a master if the master has + also acted like a slave. (Mark Miller) + +* SOLR-4498: Add list command to ZkCLI that prints out the contents of + ZooKeeper. (Roman Shaposhnik via Mark Miller) + +* SOLR-4481: SwitchQParserPlugin registered by default as 'switch' using + syntax: {!switch case=XXX case.foo=YYY case.bar=ZZZ default=QQQ}foo + (hossman) + +* SOLR-4078: Allow custom naming of SolrCloud nodes so that a new host:port + combination can take over for a previous shard. (Mark Miller) + +* SOLR-4210: Requests to a Collection that does not exist on the receiving node + should be proxied to a suitable node. (Mark Miller, Po Rui, yonik) + +* SOLR-1365: New SweetSpotSimilarityFactory allows customizable TF/IDF based + Similarity when you know the optimal "Sweet Spot" of values for the field + length and TF scoring factors. (hossman) + +* SOLR-4138: CurrencyField fields can now be used in a ValueSources to + get the "raw" value (using the default number of fractional digits) in + the default currency of the field type. There is also a new + currency(field,[CODE]) function for generating a ValueSource of the + "natural" value, converted to an optionally specified currency to + override the default for the field type. + (hossman) + +* SOLR-4503: Add REST API methods, via Restlet integration, for reading schema + elements, at /schema/fields/, /schema/dynamicfields/, /schema/fieldtypes/, + and /schema/copyfields/. (Steve Rowe) + +Bug Fixes +---------------------- + +* SOLR-2850: Do not refine facets when minCount == 1 + (Matt Smith, lundgren via Adrien Grand) + +* SOLR-4309: /browse: Improve JQuery autosuggest behavior (janhoy) + +* SOLR-4330: group.sort is ignored when using group.truncate and ex/tag + local params together (koji) + +* SOLR-4321: Collections API will sometimes use a node more than once, even + when more unused nodes are available. + (Eric Falcao, Brett Hoerner, Mark Miller) + +* SOLR-4345 : Solr Admin UI doesn't work in IE 10 (steffkes) + +* SOLR-4349 : Admin UI - Query Interface does not work in IE + (steffkes) + +* SOLR-4359: The RecentUpdates#update method should treat a problem reading the + next record the same as a problem parsing the record - log the exception and + break. (Mark Miller) + +* SOLR-4225: Term info page under schema browser shows incorrect count of terms + (steffkes) + +* SOLR-3926: Solr should support better way of finding active sorts (Eirik Lygre via + Erick Erickson) + +* SOLR-4342: Fix DataImportHandler stats to be a proper Map (hossman) + +* SOLR-3967: langid.enforceSchema option checks source field instead of target field (janhoy) + +* SOLR-4380: Replicate after startup option would not replicate until the + IndexWriter was lazily opened. (Mark Miller, Gregg Donovan) + +* SOLR-4400: Deadlock can occur in a rare race between committing and + closing a SolrIndexWriter. (Erick Erickson, Mark Miller) + +* SOLR-3655: A restarted node can briefly appear live and active before it really + is in some cases. (Mark Miller) + +* SOLR-4426: NRTCachingDirectoryFactory does not initialize maxCachedMB and maxMergeSizeMB + if <directoryFactory> is not present in solrconfig.xml (Jack Krupansky via shalin) + +* SOLR-4463: Fix SolrCoreState reference counting. (Mark Miller) + +* SOLR-4459: The Replication 'index move' rather than copy optimization doesn't + kick in when using NRTCachingDirectory or the rate limiting feature. + (Mark Miller) + +* SOLR-4421,SOLR-4165: On CoreContainer shutdown, all SolrCores should publish their + state as DOWN. (Mark Miller, Markus Jelsma) + +* SOLR-4467: Ephemeral directory implementations may not recover correctly + because the code to clear the tlog files on startup is off. (Mark Miller) + +* SOLR-4413: Fix SolrCore#getIndexDir() to return the current index directory. + (Gregg Donovan, Mark Miller) + +* SOLR-4469: A new IndexWriter must be opened on SolrCore reload when the index + directory has changed and the previous SolrCore's state should not be + propagated. (Mark Miller, Gregg Donovan) + +* SOLR-4471: Replication occurs even when a slave is already up to date. + (Mark Miller, Andre Charton) + +* SOLR-4484: ReplicationHandler#loadReplicationProperties still uses Files + rather than the Directory to try and read the replication properties files. + (Mark Miller) + +* SOLR-4352: /browse pagination now supports and preserves sort context + (Eric Spiegelberg, Erik Hatcher) + +* LUCENE-4796, SOLR-4373: Fix concurrency issue in NamedSPILoader and + AnalysisSPILoader when doing concurrent core loads in multicore + Solr configs. (Uwe Schindler, Hossman) + +* SOLR-4504: Fixed CurrencyField range queries to correctly exclude + documents w/o values (hossman) + +* SOLR-4480: A trailing + or - caused the edismax parser to throw + an exception. (Fiona Tay, Jan Høydahl, yonik) + +* SOLR-4507: The Cloud tab does not show up in the Admin UI if you + set zkHost in solr.xml. (Alfonso Presa, Mark Miller) + +* SOLR-4505: Possible deadlock around SolrCoreState update lock. + (Erick Erickson, Mark Miller) + +* SOLR-4511: When a new index is replicated into place, we need + to update the most recent replicatable index point without + doing a commit. This is important for repeater use cases, as + well as when nodes may switch master/slave roles. + (Mark Miller, Raúl Grande) + +* SOLR-4515: CurrencyField's OpenExchangeRatesOrgProvider now requires + a ratesFileLocation init param, since the previous global default + no longer works (hossman) + +* SOLR-4518: Improved CurrencyField error messages when attempting to + use a Currency that is not supported by the current JVM. (hossman) + +* SOLR-3798: Fix copyField implementation in IndexSchema to handle + dynamic field references that aren't string-equal to the name of + the referenced dynamic field. (Steve Rowe) + +* SOLR-4497: Collection Aliasing. (Mark Miller) + +Optimizations +---------------------- + +* SOLR-4339: Admin UI - Display Field-Flags on Schema-Browser + (steffkes) + +* SOLR-4340: Admin UI - Analysis's Button Spinner goes wild + (steffkes) + +* SOLR-4341: Admin UI - Plugins/Stats Page contains loooong + Values which result in horizontal Scrollbar (steffkes) + +* SOLR-3915: Color Legend for Cloud UI (steffkes) + +* SOLR-4306: Utilize indexInfo=false when gathering core names in UI + (steffkes) + +* SOLR-4284: Admin UI - make core list scrollable separate from the rest of + the UI (steffkes) + +* SOLR-4364: Admin UI - Locale based number formatting (steffkes) + +* SOLR-4521: Stop using the 'force' option for recovery replication. This + will keep some less common unnecessary replications from happening. + (Mark Miller, Simon Scofield) + +* SOLR-4529: Improve Admin UI Dashboard legibility (Felix Buenemann via + steffkes) + +* SOLR-4526: Admin UI depends on optional system info (Felix Buenemann via + steffkes) + +Other Changes +---------------------- + +* SOLR-4259: Carrot2 dependency should be declared on the mini version, not the core. + (Dawid Weiss). + +* SOLR-4348: Make the lock type configurable by system property by default. + (Mark Miller) + +* SOLR-4353: Renamed example jetty context file to reduce confusion (hossman) + +* SOLR-4384: Make post.jar report timing information (Upayavira via janhoy) + +* SOLR-4415: Add 'state' to shards (default to 'active') and read/write them to + ZooKeeper (Anshum Gupta via shalin) + +* SOLR-4394: Tests and example configs demonstrating SSL with both server + and client certs (hossman) + +* SOLR-3060: SurroundQParserPlugin highlighting tests + (Ahmet Arslan via hossman) + +* SOLR-2470: Added more tests for VelocityResponseWriter + +* SOLR-4471: Improve and clean up TestReplicationHandler. + (Amit Nithian via Mark Miller) + +* SOLR-3843: Include lucene codecs jar and enable per-field postings and docvalues + support in the schema.xml (Robert Muir, Steve Rowe) + +* SOLR-4511: Add new test for 'repeater' replication node. (Mark Miller) + +* SOLR-4458: Sort directions (asc, desc) are now case insensitive + (Shawn Heisey via hossman) + +* SOLR-2996: A bare * without a field specification is treated as *:* + by the lucene and edismax query parsers. + (hossman, Jan Høydahl, Alan Woodward, yonik) + +* SOLR-4416: Upgrade to Tika 1.3. (Markus Jelsma via Mark Miller) + +* SOLR-4200: Reduce INFO level logging from CachingDirectoryFactory + (Shawn Heisey via hossman) + +================== 4.1.0 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.2 +Carrot2 3.6.2 +Velocity 1.7 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.4.5 + +Upgrading from Solr 4.0.0 +---------------------- + +Custom java parsing plugins need to migrate from throwing the internal +ParseException to throwing SyntaxError. + +BaseDistributedSearchTestCase now randomizes the servlet context it uses when +creating Jetty instances. Subclasses that assume a hard coded context of +"/solr" should either be fixed to use the "String context" variable, or should +take advantage of the new BaseDistributedSearchTestCase(String) constructor +to explicitly specify a fixed servlet context path. See SOLR-4136 for details. + + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-2255: Enhanced pivot faceting to use local-params in the same way that + regular field value faceting can. This means support for excluding a filter + query, using a different output key, and specifying 'threads' to do + facet.method=fcs concurrently. PivotFacetHelper now extends SimpleFacet and + the getFacetImplementation() extension hook was removed. (dsmiley) + +* SOLR-3897: A highlighter parameter "hl.preserveMulti" to return all of the + values of a multiValued field in their original order when highlighting. + (Joel Bernstein via yonik) + +* SOLR-3929: Support configuring IndexWriter max thread count in solrconfig. + (phunt via Mark Miller) + +* SOLR-3906: Add support for AnalyzingSuggester (LUCENE-3842), where the + underlying analyzed form used for suggestions is separate from the returned + text. (Robert Muir) + +* SOLR-3985: ExternalFileField caches can be reloaded on firstSearcher/ + newSearcher events using the ExternalFileFieldReloader (Alan Woodward) + +* SOLR-3911: Make Directory and DirectoryFactory first class so that the majority + of Solr's features work with any custom implementations. (Mark Miller) + Additional Work: + - SOLR-4032: Files larger than an internal buffer size fail to replicate. + (Mark Miller, Markus Jelsma) + - SOLR-4033: Consistently use the solrconfig.xml lockType everywhere. + (Mark Miller, Markus Jelsma) + - SOLR-4144: Replication using too much RAM. (yonik, Markus Jelsma) + - SOLR-4187: NPE on Directory release (Mark Miller, Markus Jelsma) + +* SOLR-4051: Add <propertyWriter /> element to DIH's data-config.xml file, + allowing the user to specify the location, filename and Locale for + the "data-config.properties" file. Alternatively, users can specify their + own property writer implementation for greater control. This new configuration + element is optional, and defaults mimic prior behavior. The one exception is + that the "root" locale is default. Previously it was the machine's default locale. + (James Dyer) + +* SOLR-4084: Add FuzzyLookupFactory, which is like AnalyzingSuggester except that + it can tolerate typos in the input. (Areek Zillur via Robert Muir) + +* SOLR-4088: New and improved auto host detection strategy for SolrCloud. + (Raintung Li via Mark Miller) + +* SOLR-3970: SystemInfoHandler now exposes more details about the + JRE/VM/Java version in use. (hossman) + +* SOLR-4101: Add support for storing term offsets in the index via a + 'storeOffsetsWithPositions' flag on field definitions in the schema. + (Tom Winch, Alan Woodward) + +* SOLR-4093: Solr QParsers may now be directly invoked in the lucene + query syntax without the _query_ magic field hack. + Example: foo AND {!term f=myfield v=$qq} + (yonik) + +* SOLR-4087: Add MAX_DOC_FREQ option to MoreLikeThis. + (Andrew Janowczyk via Mark Miller) + +* SOLR-4114: Allow creating more than one shard per instance with the + Collection API. (Per Steffensen, Mark Miller) + +* SOLR-3531: Allowing configuring maxMergeSizeMB and maxCachedMB when + using NRTCachingDirectoryFactory. (Andy Laird via Mark Miller) + +* SOLR-4118: Fix replicationFactor to align with industry usage. + replicationFactor now means the total number of copies + of a document stored in the collection (or the total number of + physical indexes for a single logical slice of the collection). + For example if replicationFactor=3 then for a given shard there + will be a total of 3 replicas (one of which will normally be + designated as the leader.) (yonik) + +* SOLR-4124: You should be able to set the update log directory with the + CoreAdmin API the same way as the data directory. (Mark Miller) + +* SOLR-4028: When using ZK chroot, it would be nice if Solr would create the + initial path when it doesn't exist. (Tomás Fernández Löbbe via Mark Miller) + +* SOLR-3948: Calculate/display deleted documents in admin interface. + (Shawn Heisey via Mark Miller) + +* SOLR-4030: Allow rate limiting Directory IO based on the IO context. + (Mark Miller, Radim Kolar) + +* SOLR-4166: LBHttpSolrServer ignores ResponseParser passed in constructor. + (Steve Molloy via Mark Miller) + +* SOLR-4140: Allow access to the collections API through CloudSolrServer + without referencing an existing collection. (Per Steffensen via Mark Miller) + +* SOLR-788: Distributed search support for MLT. + (Matthew Woytowitz, Mike Anderson, Jamie Johnson, Mark Miller) + +* SOLR-4120: Collection API: Support for specifying a list of Solr addresses to + spread a new collection across. (Per Steffensen via Mark Miller) + +* SOLR-4110: Configurable Content-Type headers for PHPResponseWriters and + PHPSerializedResponseWriter. (Dominik Siebel via Mark Miller) + +* SOLR-1028: The ability to specify "transient" and "loadOnStartup" as a new properties of + <core> tags in solr.xml. Can specify "transientCacheSize" in the <cores> tag. Together + these allow cores to be loaded only when needed and only transientCacheSize transient + cores will be loaded at a time, the rest aged out on an LRU basis. + +* SOLR-4246: When update.distrib is set to skip update processors before + the distributed update processor, always include the log update processor + so forwarded updates will still be logged. (yonik) + +* SOLR-4230: The new Solr 4 spatial fields now work with the {!geofilt} and + {!bbox} query parsers. The score local-param works too. (David Smiley) + +* SOLR-1972: Add extra statistics to RequestHandlers - 5 & 15-minute reqs/sec + rolling averages; median, 75th, 95th, 99th, 99.9th percentile request times + (Alan Woodward, Shawn Heisey, Adrien Grand, Uwe Schindler) + +* SOLR-4271: Add support for PostingsHighlighter. (Robert Muir) + +* SOLR-4255: The new Solr 4 spatial fields now have a 'filter' boolean local-param + that can be set to false to not filter. It's useful when there is already a spatial + filter query but you also need to sort or boost by distance. (David Smiley) + +* SOLR-4265, SOLR-4283: Solr now parses request parameters (in URL or sent with POST + using content-type application/x-www-form-urlencoded) in its dispatcher code. It no + longer relies on special configuration settings in Tomcat or other web containers + to enable UTF-8 encoding, which is mandatory for correct Solr behaviour. Query + strings passed in via the URL need to be properly-%-escaped, UTF-8 encoded + bytes, otherwise Solr refuses to handle the request. The maximum length of + x-www-form-urlencoded POST parameters can now be configured through the + requestDispatcher/requestParsers/@formdataUploadLimitInKB setting in + solrconfig.xml (defaults to 2 MiB). Solr now works out of the box with + e.g. Tomcat, JBoss,... (Uwe Schindler, Dawid Weiss, Alex Rocher) + +* SOLR-2201: DIH's "formatDate" function now supports a timezone as an optional + fourth parameter (James Dyer, Mark Waddle) + +* SOLR-4302: New parameter 'indexInfo' (defaults to true) in CoreAdmin STATUS + command can be used to omit index specific information (Shahar Davidson via shalin) + +* SOLR-2592: Collection specific document routing. The "compositeId" + router is the default for collections with hash based routing (i.e. when + numShards=N is specified on collection creation). Documents with ids sharing + the same domain (prefix) will be routed to the same shard, allowing for + efficient querying. + + Example: + The following two documents will be indexed to the same shard + since they share the same domain "customerB!". + <code> + {"id" : "customerB!doc1" [...] } + {"id" : "customerB!doc2" [...] } + </code> + At query time, one can specify a "shard.keys" parameter that lists what + shards the query should cover. + http://.../query?q=my_query&shard.keys=customerB! + + Collections that do not specify numShards at collection creation time + use custom sharding and default to the "implicit" router. Document updates + received by a shard will be indexed to that shard, unless a "_shard_" parameter + or document field names a different shard. + (Michael Garski, Dan Rosher, yonik) + + +Optimizations +---------------------- + +* SOLR-3788: Admin Cores UI should redirect to newly created core details + (steffkes) + +* SOLR-3895: XML and XSLT UpdateRequestHandler should not try to resolve + external entities. This improves speed of loading e.g. XSL-transformed + XHTML documents. (Martin Herfurt, uschindler, hossman) + +* SOLR-3614: Fix XML parsing in XPathEntityProcessor to correctly expand + named entities, but ignore external entities. (uschindler, hossman) + +* SOLR-3734: Improve Schema-Browser Handling for CopyField using + dynamicField's (steffkes) + +* SOLR-3941: The "commitOnLeader" part of distributed recovery can use + openSearcher=false. (Tomás Fernández Löbbe via Mark Miller) + +* SOLR-4063: Allow CoreContainer to load multiple SolrCores in parallel rather + than just serially. (Mark Miller) + +* SOLR-4199: When doing zk retries due to connection loss, rather than just + retrying for 2 minutes, retry in proportion to the session timeout. + (Mark Miller) + +* SOLR-4262: Replication Icon on Dashboard does not reflect Master-/Slave- + State (steffkes) + +* SOLR-4264: Missing Error-Screen on UI's Cloud-Page (steffkes) + +* SOLR-4261: Percentage Infos on Dashboard have a fixed width (steffkes) + +* SOLR-3851: create a new core/delete an existing core should also update + the main/left list of cores on the admin UI (steffkes) + +* SOLR-3840: XML query response display is unreadable in Solr Admin Query UI + (steffkes) + +* SOLR-3982: Admin UI: Various Dataimport Improvements (steffkes) + +* SOLR-4296: Admin UI: Improve Dataimport Auto-Refresh (steffkes) + +* SOLR-3458: Allow multiple Items to stay open on Plugins-Page (steffkes) + +Bug Fixes +---------------------- + +* SOLR-4288: Improve logging for FileDataSource (basePath, relative + resources). (Dawid Weiss) + +* SOLR-4007: Morfologik dictionaries not available in Solr field type + due to class loader lookup problems. (Lance Norskog, Dawid Weiss) + +* SOLR-3560: Handle different types of Exception Messages for Logging UI + (steffkes) + +* SOLR-3637: Commit Status at Core-Admin UI is always false (steffkes) + +* SOLR-3917: Partial State on Schema-Browser UI is not defined for Dynamic + Fields & Types (steffkes) + +* SOLR-3939: Consider a sync attempt from leader to replica that fails due + to 404 a success. (Mark Miller, Joel Bernstein) + +* SOLR-3940: Rejoining the leader election incorrectly triggers the code path + for a fresh cluster start rather than fail over. (Mark Miller) + +* SOLR-3961: Fixed error using LimitTokenCountFilterFactory + (Jack Krupansky, hossman) + +* SOLR-3933: Distributed commits are not guaranteed to be ordered within a + request. (Mark Miller) + +* SOLR-3939: An empty or just replicated index cannot become the leader of a + shard after a leader goes down. (Joel Bernstein, yonik, Mark Miller) + +* SOLR-3971: A collection that is created with numShards=1 turns into a + numShards=2 collection after starting up a second core and not specifying + numShards. (Mark Miller) + +* SOLR-3988: Fixed SolrTestCaseJ4.adoc(SolrInputDocument) to respect + field and document boosts (hossman) + +* SOLR-3981: Fixed bug that resulted in document boosts being compounded in + <copyField/> destination fields. (hossman) + +* SOLR-3920: Fix server list caching in CloudSolrServer when using more than one + collection list with the same instance. (Grzegorz Sobczyk, Mark Miller) + +* SOLR-3938: prepareCommit command omits commitData causing a failure to trigger + replication to slaves. (yonik) + +* SOLR-3992: QuerySenderListener doesn't populate document cache. + (Shotaro Kamio, yonik) + +* SOLR-3995: Recovery may never finish on SolrCore shutdown if the last reference to + a SolrCore is closed by the recovery process. (Mark Miller) + +* SOLR-3998: Atomic update on uniqueKey field itself causes duplicate document. + (Eric Spencer, yonik) + +* SOLR-4001: In CachingDirectoryFactory#close, if there are still refs for a + Directory outstanding, we need to wait for them to be released before closing. + (Mark Miller) + +* SOLR-4005: If CoreContainer fails to register a created core, it should close it. + (Mark Miller) + +* SOLR-4009: OverseerCollectionProcessor is not resilient to many error conditions + and can stop running on errors. (Raintung Li, milesli, Mark Miller) + +* SOLR-4019: Log stack traces for 503/Service Unavailable SolrException if not + thrown by PingRequestHandler. Do not log exceptions if a user tries to view a + hidden file using ShowFileRequestHandler. (Tomás Fernández Löbbe via James Dyer) + +* SOLR-3589: Edismax parser does not honor mm parameter if analyzer splits a token. + (Tom Burton-West, Robert Muir) + +* SOLR-4031: Upgrade to Jetty 8.1.7 to fix a bug where in very rare occasions + the content of two concurrent requests get mixed up. (Per Steffensen, yonik) + +* SOLR-4060: ReplicationHandler can try and do a snappull and open a new IndexWriter + after shutdown has already occurred, leaving an IndexWriter that is not closed. + (Mark Miller) + +* SOLR-4055: Fix a thread safety issue with the Collections API that could + cause actions to be targeted at the wrong SolrCores. + (Raintung Li, Per Steffensen via Mark Miller) + +* SOLR-3993: If multiple SolrCore's for a shard coexist on a node, on cluster + restart, leader election would stall until timeout, waiting to see all of + the replicas come up. (Mark Miller, Alexey Kudinov) + +* SOLR-2045: Databases that require a commit to be issued before closing the + connection on a non-read-only database leak connections. Also expanded the + SqlEntityProcessor test to sometimes use Derby as well as HSQLDB (Derby is + one db affected by this bug). (Fenlor Sebastia, James Dyer) + +* SOLR-4064: When there is an unexpected exception while trying to run the new + leader process, the SolrCore will not correctly rejoin the election. + (Po Rui via Mark Miller) + +* SOLR-3989: SolrZkClient constructor dropped exception cause when throwing + a new RuntimeException. (Colin Bartolome, yonik) + +* SOLR-4036: field aliases in fl should not cause properties of target field + to be used. (Martin Koch, yonik) + +* SOLR-4003: The SolrZKClient clean method should not try and clear zk paths + that start with /zookeeper, as this can fail and stop the removal of + further nodes. (Mark Miller) + +* SOLR-4076: SolrQueryParser should run fuzzy terms through + MultiTermAwareComponents to ensure that (for example) a fuzzy query of + foobar~2 is equivalent to FooBar~2 on a field that includes lowercasing. + (yonik) + +* SOLR-4081: QueryParsing.toString, used during debugQuery=true, did not + correctly handle ExtendedQueries such as WrappedQuery + (used when cache=false), spatial queries, and frange queries. + (Eirik Lygre, yonik) + +* SOLR-3959: Ensure the internal comma separator of poly fields is escaped + for CSVResponseWriter. (Areek Zillur via Robert Muir) + +* SOLR-4075: A logical shard that has had all of its SolrCores unloaded should + be removed from the cluster state. (Mark Miller, Gilles Comeau) + +* SOLR-4034: Check if a collection already exists before trying to create a + new one. (Po Rui, Mark Miller) + +* SOLR-4097: Race can cause NPE in logging line on first cluster state update. + (Mark Miller) + +* SOLR-4099: Allow the collection api work queue to make forward progress even + when its watcher is not fired for some reason. (Raintung Li via Mark Miller) + +* SOLR-3960: Fixed a bug where Distributed Grouping ignored PostFilters + (Nathan Visagan, hossman) + +* SOLR-3842: DIH would not populate multivalued fields if the column name + derives from a resolved variable (James Dyer) + +* SOLR-4117: Retrieving the size of the index may use the wrong index dir if + you are replicating. + (Mark Miller, Markus Jelsma) + +* SOLR-2890: Fixed a bug that prevented omitNorms and omitTermFreqAndPositions + options from being respected in some <fieldType/> declarations (hossman) + +* SOLR-4159: When we are starting a shard from rest, a potential leader should + not consider its last published state when deciding if it can be the new + leader. (Mark Miller) + +* SOLR-4158: When a core is registering in ZooKeeper it may not wait long + enough to find the leader due to how long the potential leader waits to see + replicas. (Mark Miller, Alain Rogister) + +* SOLR-4162: ZkCli usage examples are not correct because the zkhost parameter + is not present and it is mandatory for all commands. + (Tomás Fernández Löbbe via Mark Miller) + +* SOLR-4071: Validate that name is pass to Collections API create, and behave the + same way as on startup when collection.configName is not explicitly passed. + (Po Rui, Mark Miller) + +* SOLR-4127: Added explicit error message if users attempt Atomic document + updates with either updateLog or DistribUpdateProcessor. (hossman) + +* SOLR-4136: Fix SolrCloud behavior when using "hostContext" containing "_" + or"/" characters. This fix also makes SolrCloud more accepting of + hostContext values with leading/trailing slashes. (hossman) + +* SOLR-4168: Ensure we are using the absolute latest index dir when getting + list of files for replication. (Mark Miller) + +* SOLR-4171: CachingDirectoryFactory should not return any directories after it + has been closed. (Mark Miller) + +* SOLR-4102: Fix UI javascript error if canonical hostname can not be resolved + (steffkes via hossman) + +* SOLR-4178: ReplicationHandler should abort any current pulls and wait for + its executor to stop during core close. (Mark Miller) + +* SOLR-3918: Fixed the 'dist-war-excl-slf4j' ant target to exclude all + slf4j jars, so that the resulting war is usable as is provided the servlet + container includes the correct slf4j api and impl jars. + (Shawn Heisey, hossman) + +* SOLR-4198: OverseerCollectionProcessor should implement ClosableThread. + (Mark Miller) + +* SOLR-4213: Directories that are not shutdown until DirectoryFactory#close + do not have close listeners called on them. (Mark Miller) + +* SOLR-4134: Standard (XML) request writer cannot "set" multiple values into + multivalued field with partial updates. (Luis Cappa Banda, Will Butler, shalin) + +* SOLR-3972: Fix ShowFileRequestHandler to not log a warning in the + (expected) situation of a file not found. (hossman) + +* SOLR-4133: Cannot "set" field to null with partial updates when using the + standard RequestWriter. (Will Butler, shalin) + +* SOLR-4223: "maxFormContentSize" in jetty.xml is not picked up by jetty 8 + so set it via solr webapp context file. (shalin) + +* SOLR-4175:SearchComponent chain can't contain two components of the + same class and use debugQuery. (Tomás Fernández Löbbe via ehatcher) + +* SOLR-4244: When coming back from session expiration we should not wait for + the leader to see us in the down state if we are the node that must become + the leader. (Mark Miller) + +* SOLR-4245: When a core is registering with ZooKeeper, the timeout to find the + leader in the cluster state is 30 seconds rather than leaderVoteWait + extra + time. (Mark Miller) + +* SOLR-4238: Fix jetty example requestLog config (jm via hossman) + +* SOLR-4251: Fix SynonymFilterFactory when an optional tokenizerFactory is supplied. + (Chris Bleakley via rmuir) + +* SOLR-4253: Misleading resource loading warning from Carrot2 clustering + component fixed (Stanisław Osiński) + +* SOLR-4257: PeerSync updates and Log Replay updates should not wait for + a ZooKeeper connection in order to proceed. (yonik) + +* SOLR-4045: SOLR admin page returns HTTP 404 on core names containing + a '.' (dot) (steffkes) + +* SOLR-4176: analysis ui: javascript not properly handling URL decoding + of input (steffkes) + +* SOLR-4079: Long core names break web gui appearance and functionality + (steffkes) + +* SOLR-4263: Incorrect Link from Schema-Browser to Query From for Top-Terms + (steffkes) + +* SOLR-3829: Admin UI Logging events broken if schema.xml defines a catch-all + dynamicField with type ignored (steffkes) + +* SOLR-4275: Fix TrieTokenizer to no longer throw StringIndexOutOfBoundsException + in admin UI / AnalysisRequestHandler when you enter no number to tokenize. + (Uwe Schindler) + +* SOLR-4279: Wrong exception message if _version_ field is multivalued (shalin) + +* SOLR-4170: The 'backup' ReplicationHandler command can sometimes use a stale + index directory rather than the current one. (Mark Miller, Marcin Rzewucki) + +* SOLR-3876: Solr Admin UI is completely dysfunctional on IE 9 (steffkes) + +* SOLR-4112: Fixed DataImportHandler ZKAwarePropertiesWriter implementation so + import works fine with SolrCloud clusters (Deniz Durmus, James Dyer, + Erick Erickson, shalin) + +* SOLR-4291: Harden the Overseer work queue thread loop. (Mark Miller) + +* SOLR-3820: Solr Admin Query form is missing some edismax request parameters + (steffkes) + +* SOLR-4217: post.jar no longer ignores -Dparams when -Durl is used. + (Alexandre Rafalovitch, ehatcher) + +* SOLR-4303: On replication, if the generation of the master is lower than the + slave we need to force a full copy of the index. (Mark Miller, Gregg Donovan) + +* SOLR-4266: HttpSolrServer does not release connection properly on exception + when no response parser is used. (Steve Molloy via Mark Miller) + +* SOLR-2298: Updated JavaDoc for SolrDocument.addField and SolrInputDocument.addField + to have more information on name and value parameters. (Siva Natarajan) + +Other Changes +---------------------- + +* SOLR-4106: Javac/ ivy path warnings with morfologik fixed by + upgrading to Morfologik 1.5.5 (Robert Muir, Dawid Weiss) + +* SOLR-3899: SolrCore should not log at warning level when the index directory + changes - it's an info event. (Tobias Bergman, Mark Miller) + +* SOLR-3861: Refactor SolrCoreState so that it's managed by SolrCore. + (Mark Miller, hossman) + +* SOLR-3966: Eliminate superfluous warning from LanguageIdentifierUpdateProcessor + (Markus Jelsma via hossman) + +* SOLR-3932: SolrCmdDistributorTest either takes 3 seconds or 3 minutes. + (yonik, Mark Miller) + +* SOLR-3856: New tests for SqlEntityProcessor/CachedSqlEntityProcessor + (James Dyer) + +* SOLR-4067: ZkStateReader#getLeaderProps should not return props for a leader + that it does not think is live. (Mark Miller) + +* SOLR-4086: DIH refactor of VariableResolver and Evaluator. VariableResolver + and each built-in Evaluator are separate concrete classes. DateFormatEvaluator + now defaults with the ROOT Locale. However, users may specify a different + Locale using an optional new third parameter. (James Dyer) + +* SOLR-3602: Update ZooKeeper to 3.4.5 (Mark Miller) + +* SOLR-4095: DIH NumberFormatTransformer & DateFormatTransformer default to the + ROOT Locale if none is specified. These previously used the machine's default. + (James Dyer) + +* SOLR-4096: DIH FileDataSource & FieldReaderDataSource default to UTF-8 encoding + if none is specified. These previously used the machine's default. + (James Dyer) + +* SOLR-1916: DIH to not use Lucene-forbidden Java APIs + (default encoding, locale, etc.) (James Dyer, Robert Muir) + +* SOLR-4111: SpellCheckCollatorTest#testContextSensitiveCollate to test against + both DirectSolrSpellChecker & IndexBasedSpellChecker + (Tomás Fernández Löbbe via James Dyer) + +* SOLR-2141: Better test coverage for Evaluators (James Dyer) + +* SOLR-4119: Update Guava to 13.0.1 (Mark Miller) + +* SOLR-4074: Raise default ramBufferSizeMB to 100 from 32. + (yonik, Mark Miller) + +* SOLR-4062: The update log location in solrconfig.xml should default to + ${solr.ulog.dir} rather than ${solr.data.dir:} (Mark Miller) + +* SOLR-4155: Upgrade Jetty to 8.1.8. (Robert Muir) + +* SOLR-2986: Add MoreLikeThis to warning about features that require uniqueKey. + Also, change the warning to warn log level. (Shawn Heisey via Mark Miller) + +* SOLR-4163: README improvements (Shawn Heisey via hossman) + +* SOLR-4248: "ant eclipse" should declare .svn directories as derived. + (Shawn Heisey via Mark Miller) + +* SOLR-3279: Upgrade Carrot2 to 3.6.2 (Stanisław Osiński) + +* SOLR-4254: Harden the 'leader requests replica to recover' code path. + (Mark Miller, yonik) + +* SOLR-4226: Extract fl parsing code out of ReturnFields constructor. + (Ryan Ernst via Robert Muir) + +* SOLR-4208: ExtendedDismaxQParserPlugin has been refactored to make + subclassing easier. (Tomás Fernández Löbbe, hossman) + +* SOLR-3735: Relocate the example mime-to-extension mapping, and + upgrade Velocity Engine to 1.7 (ehatcher) + +* SOLR-4287: Removed "apache-" prefix from Solr distribution and artifact + filenames. (Ryan Ernst, Robert Muir, Steve Rowe) + +* SOLR-4016: Deduplication does not work with atomic/partial updates so + disallow atomic update requests which change signature generating fields. + (Joel Nothman, yonik, shalin) + +* SOLR-4308: Remove the problematic and now unnecessary log4j-over-slf4j. + (Mark Miller) + +================== 4.0.0 ================== + +Versions of Major Components +--------------------- +Apache Tika 1.2 +Carrot2 3.5.0 +Velocity 1.6.4 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.3.6 + +Upgrading from Solr 4.0.0-BETA +---------------------- + +In order to better support distributed search mode, the TermVectorComponent's +response format has been changed so that if the schema defines a +uniqueKeyField, then that field value is used as the "key" for each document in +its response section, instead of the internal lucene doc id. Users w/o a +uniqueKeyField will continue to see the same response format. See SOLR-3229 +for more details. + +If you are using SolrCloud's distributed update request capabilities and a non +string type id field, you must re-index. + +Upgrading from Solr 4.0.0-ALPHA +---------------------- + +Solr is now much more strict about requiring that the uniqueKeyField feature +(if used) must refer to a field which is not multiValued. If you upgrade from +an earlier version of Solr and see an error that your uniqueKeyField "can not +be configured to be multivalued" please add 'multiValued="false"' to the +<field /> declaration for your uniqueKeyField. See SOLR-3682 for more details. + +In addition, please review the notes above about upgrading from 4.0.0-BETA + +Upgrading from Solr 3.6 +---------------------- + +* The Lucene index format has changed and as a result, once you upgrade, + previous versions of Solr will no longer be able to read your indices. + In a master/slave configuration, all searchers/slaves should be upgraded + before the master. If the master were to be updated first, the older + searchers would not be able to read the new index format. + +* Setting abortOnConfigurationError=false is no longer supported + (since it has never worked properly). Solr will now warn you if + you attempt to set this configuration option at all. (see SOLR-1846) + +* The default logic for the 'mm' param of the 'dismax' QParser has + been changed. If no 'mm' param is specified (either in the query, + or as a default in solrconfig.xml) then the effective value of the + 'q.op' param (either in the query or as a default in solrconfig.xml + or from the 'defaultOperator' option in schema.xml) is used to + influence the behavior. If q.op is effectively "AND" then mm=100%. + If q.op is effectively "OR" then mm=0%. Users who wish to force the + legacy behavior should set a default value for the 'mm' param in + their solrconfig.xml file. + +* The VelocityResponseWriter is no longer built into the core. Its JAR and + dependencies now need to be added (via <lib> or solr/home lib inclusion), + and it needs to be registered in solrconfig.xml like this: + <queryResponseWriter name="velocity" class="solr.VelocityResponseWriter"/> + +* The update request parameter to choose Update Request Processor Chain is + renamed from "update.processor" to "update.chain". The old parameter was + deprecated but still working since Solr3.2, but is now removed + entirely. + +* The <indexDefaults> and <mainIndex> sections of solrconfig.xml are discontinued + and replaced with the <indexConfig> section. There are also better defaults. + When migrating, if you don't know what your old settings mean, simply delete + both <indexDefaults> and <mainIndex> sections. If you have customizations, + put them in <indexConfig> section - with same syntax as before. + +* Two of the SolrServer subclasses in SolrJ were renamed/replaced. + CommonsHttpSolrServer is now HttpSolrServer, and + StreamingUpdateSolrServer is now ConcurrentUpdateSolrServer. + +* The PingRequestHandler no longer looks for a <healthcheck/> option in the + (legacy) <admin> section of solrconfig.xml. Users who wish to take + advantage of this feature should configure a "healthcheckFile" init param + directly on the PingRequestHandler. As part of this change, relative file + paths have been fixed to be resolved against the data dir. See the example + solrconfig.xml and SOLR-1258 for more details. + +* Due to low level changes to support SolrCloud, the uniqueKey field can no + longer be populated via <copyField/> or <field default=...> in the + schema.xml. Users wishing to have Solr automatically generate a uniqueKey + value when adding documents should instead use an instance of + solr.UUIDUpdateProcessorFactory in their update processor chain. See + SOLR-2796 for more details. + +In addition, please review the notes above about upgrading from 4.0.0-BETA, and 4.0.0-ALPHA + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-3670: New CountFieldValuesUpdateProcessorFactory makes it easy to index + the number of values in another field for later use at query time. (hossman) + +* SOLR-2768: new "mod(x,y)" function for computing the modulus of two value + sources. (hossman) + +* SOLR-3238: Numerous small improvements to the Admin UI (steffkes) + +* SOLR-3597: seems like a lot of wasted whitespace at the top of the admin screens + (steffkes) + +* SOLR-3304: Added Solr adapters for Lucene 4's new spatial module. With + SpatialRecursivePrefixTreeFieldType ("location_rpt" in example schema), it is + possible to index a variable number of points per document (and sort on them), + index not just points but any Spatial4j supported shape such as polygons, and + to query on these shapes too. Polygons requires adding JTS to the classpath. + (David Smiley) + +* SOLR-3825: Added optional capability to log what ids are in a response + (Scott Stults via gsingers) + +* SOLR-3821: Added 'df' to the UI Query form (steffkes) + +* SOLR-3822: Added hover titles to the edismax params on the UI Query form + (steffkes) + +Optimizations +---------------------- + +* SOLR-3715: improve concurrency of the transaction log by removing + synchronization around log record serialization. (yonik) + +* SOLR-3807: Currently during recovery we pause for a number of seconds after + waiting for the leader to see a recovering state so that any previous updates + will have finished before our commit on the leader - we don't need this wait + for peersync. (Mark Miller) + +* SOLR-3837: When a leader is elected and asks replicas to sync back to him and + that fails, we should ask those nodes to recovery asynchronously rather than + synchronously. (Mark Miller) + +* SOLR-3709: Cache the url list created from the ClusterState in CloudSolrServer + on each request. (Mark Miller) + + +Bug Fixes +---------------------- + +* SOLR-3685: Solr Cloud sometimes skipped peersync attempt and replicated instead due + to tlog flags not being cleared when no updates were buffered during a previous + replication. (Markus Jelsma, Mark Miller, yonik) + +* SOLR-3229: Fixed TermVectorComponent to work with distributed search + (Hang Xie, hossman) + +* SOLR-3725: Fixed package-local-src-tgz target to not bring in unnecessary jars + and binary contents. (Michael Dodsworth via rmuir) + +* SOLR-3649: Fixed bug in JavabinLoader that caused deleteById(List<String> ids) + to not work in SolrJ (siren) + +* SOLR-3730: Rollback is not implemented quite right and can cause corner case fails in + SolrCloud tests. (rmuir, Mark Miller) + +* SOLR-2981: Fixed StatsComponent to no longer return duplicated information + when requesting multiple stats.facet fields. + (Roman Kliewer via hossman) + +* SOLR-3743: Fixed issues with atomic updates and optimistic concurrency in + conjunction with stored copyField targets by making real-time get never + return copyField targets. (yonik) + +* SOLR-3746: Proper error reporting if updateLog is configured w/o necessary + "_version_" field in schema.xml (hossman) + +* SOLR-3745: Proper error reporting if SolrCloud mode is used w/o + necessary "_version_" field in schema.xml (hossman) + +* SOLR-3770: Overseer may lose updates to cluster state (siren) + +* SOLR-3721: Fix bug that could theoretically allow multiple recoveries to run + briefly at the same time if the recovery thread join call was interrupted. + (Per Steffensen, Mark Miller) + +* SOLR-3782: A leader going down while updates are coming in can cause shard + inconsistency. (Mark Miller) + +* SOLR-3611: We do not show ZooKeeper data in the UI for a node that has children. + (Mark Miller) + +* SOLR-3789: Fix bug in SnapPuller that caused "internal" compression to fail. + (siren) + +* SOLR-3790: ConcurrentModificationException could be thrown when using hl.fl=*. + Fixed in r1231606. (yonik, koji) + +* SOLR-3668: DataImport : Specifying Custom Parameters (steffkes) + +* SOLR-3793: UnInvertedField faceting cached big terms in the filter + cache that ignored deletions, leading to duplicate documents in search + later when a filter of the same term was specified. + (Günter Hipler, hossman, yonik) + +* SOLR-3679: Core Admin UI gives no feedback if "Add Core" fails (steffkes, hossman) + +* SOLR-3795: Fixed LukeRequestHandler response to correctly return field name + strings in copyDests and copySources arrays (hossman) + +* SOLR-3699: Fixed some Directory leaks when there were errors during SolrCore + or SolrIndexWriter initialization. (hossman) + +* SOLR-3518: Include final 'hits' in log information when aggregating a + distributed request (Markus Jelsma via hossman) + +* SOLR-3628: SolrInputField and SolrInputDocument are now consistently backed + by Collections passed in to setValue/setField, and defensively copy values + from Collections passed to addValue/addField + (Tom Switzer via hossman) + +* SOLR-3595: CurrencyField now generates an appropriate error on schema init + if it is configured as multiValued - this has never been properly supported, + but previously failed silently in odd ways. (hossman) + +* SOLR-3823: Fix 'bq' parsing in edismax. Please note that this required + reverting the negative boost support added by SOLR-3278 (hossman) + +* SOLR-3827: Fix shareSchema=true in solr.xml + (Tomás Fernández Löbbe via hossman) + +* SOLR-3809: Fixed config file replication when subdirectories are used + (Emmanuel Espina via hossman) + +* SOLR-3828: Fixed QueryElevationComponent so that using 'markExcludes' does + not modify the result set or ranking of 'excluded' documents relative to + not using elevation at all. (Alexey Serba via hossman) + +* SOLR-3569: Fixed debug output on distributed requests when there are no + results found. (David Bowen via hossman) + +* SOLR-3811: Query Form using wrong values for dismax, edismax (steffkes) + +* SOLR-3779: DataImportHandler's LineEntityProcessor when used in conjunction + with FileListEntityProcessor would only process the first file. + (Ahmet Arslan via James Dyer) + +* SOLR-3791: CachedSqlEntityProcessor would throw a NullPointerException when + a query returns a row with a NULL key. (Steffen Moelter via James Dyer) + +* SOLR-3833: When a election is started because a leader went down, the new + leader candidate should decline if the last state they published was not + active. (yonik, Mark Miller) + +* SOLR-3836: When doing peer sync, we should only count sync attempts that + cannot reach the given host as success when the candidate leader is + syncing with the replicas - not when replicas are syncing to the leader. + (Mark Miller) + +* SOLR-3835: In our leader election algorithm, if on connection loss we found + we did not create our election node, we should retry, not throw an exception. + (Mark Miller) + +* SOLR-3834: A new leader on cluster startup should also run the leader sync + process in case there was a bad cluster shutdown. (Mark Miller) + +* SOLR-3772: On cluster startup, we should wait until we see all registered + replicas before running the leader process - or if they all do not come up, + N amount of time. (Mark Miller) + +* SOLR-3756: If we are elected the leader of a shard, but we fail to publish + this for any reason, we should clean up and re trigger a leader election. + (Mark Miller) + +* SOLR-3812: ConnectionLoss during recovery can cause lost updates, leading to + shard inconsistency. (Mark Miller) + +* SOLR-3813: When a new leader syncs, we need to ask all shards to sync back, + not just those that are active. (Mark Miller) + +* SOLR-3641: CoreContainer is not persisting roles core attribute. + (hossman, Mark Miller) + +* SOLR-3527: SolrCmdDistributor drops some of the important commit attributes + (maxOptimizeSegments, softCommit, expungeDeletes) when sending a commit to + replicas. (Andy Laird, Tomás Fernández Löbbe, Mark Miller) + +* SOLR-3844: SolrCore reload can fail because it tries to remove the index + write lock while already holding it. (Mark Miller) + +* SOLR-3831: Atomic updates do not distribute correctly to other nodes. + (Jim Musil, Mark Miller) + +* SOLR-3465: Replication causes two searcher warmups. + (Michael Garski, Mark Miller) + +* SOLR-3645: /terms should default to distrib=false. (Nick Cotton, Mark Miller) + +* SOLR-3759: Various fixes to the example-DIH configs (Ahmet Arslan, hossman) + +* SOLR-3777: Dataimport-UI does not send unchecked checkboxes (Glenn MacStravic + via steffkes) + +* SOLR-3850: DataImportHandler "cacheKey" parameter was incorrectly renamed "cachePk" + (James Dyer) + +* SOLR-3087: Fixed DOMUtil so that code doing attribute validation will + automatically ignore nodes in the reserved "xml" prefix - in particular this + fixes some bugs related to xinclude and fieldTypes. + (Amit Nithian, hossman) + +* SOLR-3783: Fixed Pivot Faceting to work with facet.missing=true (hossman) + +* SOLR-3869: A PeerSync attempt to its replicas by a candidate leader should + not fail on o.a.http.conn.ConnectTimeoutException. (Mark Miller) + +* SOLR-3875: Fixed index boosts on multi-valued fields when docBoost is used + (hossman) + +* SOLR-3878: Exception when using open-ended range query with CurrencyField (janhoy) + +* SOLR-3891: CacheValue in CachingDirectoryFactory cannot be used outside of + solr.core package. (phunt via Mark Miller) + +* SOLR-3892: Inconsistent locking when accessing cache in CachingDirectoryFactory + from RAMDirectoryFactory and MockDirectoryFactory. (phunt via Mark Miller) + +* SOLR-3883: Distributed indexing forwards non-applicable request params. + (Dan Sutton, Per Steffensen, yonik, Mark Miller) + +* SOLR-3903: Fixed MissingFormatArgumentException in ConcurrentUpdateSolrServer + (hossman) + +* SOLR-3916: Fixed whitespace bug in parsing the fl param (hossman) + +Other Changes +---------------------- + +* SOLR-3690: Fixed binary release packages to include dependencies needed for + the solr-test-framework (hossman) + +* SOLR-2857: The /update/json and /update/csv URLs were restored to aid + in the migration of existing clients. (yonik) + +* SOLR-3691: SimplePostTool: Mode for crawling/posting web pages + See http://wiki.apache.org/solr/ExtractingRequestHandler for examples (janhoy) + +* SOLR-3707: Upgrade Solr to Tika 1.2 (janhoy) + +* SOLR-2747: Updated changes2html.pl to handle Solr's CHANGES.txt; added + target 'changes-to-html' to solr/build.xml. + (Steve Rowe, Robert Muir) + +* SOLR-3752: When a leader goes down, have the Overseer clear the leader state + in cluster.json (Mark Miller) + +* SOLR-3751: Add defensive checks for SolrCloud updates and requests that ensure + the local state matches what we can tell the request expected. (Mark Miller) + +* SOLR-3773: Hash based on the external String id rather than the indexed + representation for distributed updates. (Michael Garski, yonik, Mark Miller) + +* SOLR-3780: Maven build: Make solrj tests run separately from solr-core. + (Steve Rowe) + +* SOLR-3772: Optionally, on cluster startup, we can wait until we see all registered + replicas before running the leader process - or if they all do not come up, + N amount of time. (Jan Høydahl, Per Steffensen, Mark Miller) + +* SOLR-3750: Optionally, on session expiration, we can explicitly wait some time before + running the leader sync process so that we are sure every node participates. + (Per Steffensen, Mark Miller) + +* SOLR-3824: Velocity: Error messages from search not displayed (janhoy) + +* SOLR-3826: Test framework improvements for specifying coreName on initCore + (Amit Nithian, hossman) + +* SOLR-3749: Allow default UpdateLog syncLevel to be configured by + solrconfig.xml (Raintung Li, Mark Miller) + +* SOLR-3845: Rename numReplicas to replicationFactor in Collections API. + (yonik, Mark Miller) + +* SOLR-3815: SolrCloud - Add properties such as "range" to shards, which changes + the clusterstate.json and puts the shard replicas under "replicas". (yonik) + +* SOLR-3871: SyncStrategy should use an executor for the threads it creates to + request recoveries. (Mark Miller) + +* SOLR-3870: SyncStrategy should have a close so it can abort earlier on + shutdown. (Mark Miller) + + +================== 4.0.0-BETA =================== + + +Versions of Major Components +--------------------- +Apache Tika 1.1 +Carrot2 3.5.0 +Velocity 1.6.4 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.3.6 + +Upgrading from Solr 4.0.0-ALPHA +---------------------- + +Solr is now much more strict about requiring that the uniqueKeyField feature +(if used) must refer to a field which is not multiValued. If you upgrade from +an earlier version of Solr and see an error that your uniqueKeyField "can not +be configured to be multivalued" please add 'multiValued="false"' to the +<field /> declaration for your uniqueKeyField. See SOLR-3682 for more details. + +Detailed Change List +---------------------- + +New Features +---------------------- + +* LUCENE-4201: Added JapaneseIterationMarkCharFilterFactory to normalize Japanese + iteration marks. (Robert Muir, Christian Moen) + +* SOLR-1856: In Solr Cell, literals should override Tika-parsed values. + Patch adds a param "literalsOverride" which defaults to true, but can be set + to "false" to let Tika-parsed values be appended to literal values (Chris Harris, janhoy) + +* SOLR-3488: Added a Collection management API for SolrCloud. + (Tommaso Teofili, Sami Siren, yonik, Mark Miller) + +* SOLR-3559: Full deleteByQuery support with SolrCloud distributed indexing. + All replicas of a shard will be consistent, even if updates arrive in a + different order on different replicas. (yonik) + +* SOLR-1929: Index encrypted documents with ExtractingUpdateRequestHandler. + By supplying resource.password=<mypw> or specifying an external file with regular + expressions matching file names, Solr will decrypt and index PDFs and DOCX formats. + (janhoy, Yiannis Pericleous) + +* SOLR-3562: Add options to remove instance dir or data dir on core unload. + (Mark Miller, Per Steffensen) + +* SOLR-2702: The default directory factory was changed to NRTCachingDirectoryFactory + which wraps the StandardDirectoryFactory and caches small files for improved + Near Real-time (NRT) performance. (Mark Miller, yonik) + +* SOLR-2616: Include a sample java util logging configuration file. + (David Smiley, Mark Miller) + +* SOLR-3460: Add cloud-scripts directory and a zkcli.sh|bat tool for easy scripting + and interaction with ZooKeeper. (Mark Miller) + +* SOLR-1725: StatelessScriptUpdateProcessorFactory allows users to implement + the full ScriptUpdateProcessor API using any scripting language with a + javax.script.ScriptEngineFactory + (Uri Boness, ehatcher, Simon Rosenthal, hossman) + +* SOLR-139: Change to updateable documents to create the document if it doesn't + already exist. To assert that the document must exist, use the optimistic + concurrency feature by specifying a _version_ of 1. (yonik) + +* LUCENE-2510, LUCENE-4044: Migrated Solr's Tokenizer-, TokenFilter-, and + CharFilterFactories to the lucene-analysis module. To add new analysis + modules to Solr (like ICU, SmartChinese, Morfologik,...), just drop in + the JAR files from Lucene's binary distribution into your Solr instance's + lib folder. The factories are automatically made available with SPI. + (Chris Male, Robert Muir, Uwe Schindler) + +* SOLR-3634, SOLR-3635: CoreContainer and CoreAdminHandler will now remember + and report back information about failures to initialize SolrCores. These + failures will be accessible from the web UI and CoreAdminHandler STATUS + command until they are "reset" by creating/renaming a SolrCore with the + same name. (hossman, steffkes) + +* SOLR-1280: Added commented-out example of the new script update processor + to the example configuration. See http://wiki.apache.org/solr/ScriptUpdateProcessor (ehatcher) + +* SOLR-3672: SimplePostTool: Improvements for posting files + Support for auto mode, recursive and wildcards (janhoy) + +Optimizations +---------------------- + +* SOLR-3708: Add hashCode to ClusterState so that structures built based on the + ClusterState can be easily cached. (Mark Miller) + +* SOLR-3709: Cache the url list created from the ClusterState in CloudSolrServer on each + request. (Mark Miller, yonik) + +* SOLR-3710: Change CloudSolrServer so that update requests are only sent to leaders by + default. (Mark Miller) + +Bug Fixes +---------------------- + +* SOLR-3582: Our ZooKeeper watchers respond to session events as if they are change events, + creating undesirable side effects. (Trym R. Møller, Mark Miller) + +* SOLR-3467: ExtendedDismax escaping is missing several reserved characters + (Michael Dodsworth via janhoy) + +* SOLR-3587: After reloading a SolrCore, the original Analyzer is still used rather than a new + one. (Alexey Serba, yonik, rmuir, Mark Miller) + +* LUCENE-4185: Fix a bug where CharFilters were wrongly being applied twice. (Michael Froh, rmuir) + +* SOLR-3610: After reloading a core, indexing would fail on any newly added fields to the schema. (Brent Mills, rmuir) + +* SOLR-3377: edismax fails to correctly parse a fielded query wrapped by parens. + This regression was introduced in 3.6. (Bernd Fehling, Jan Høydahl, yonik) + +* SOLR-3621: Fix rare concurrency issue when opening a new IndexWriter for replication or rollback. + (Mark Miller) + +* SOLR-1781: Replication index directories not always cleaned up. + (Markus Jelsma, Terje Sten Bjerkseth, Mark Miller) + +* SOLR-3639: Update ZooKeeper to 3.3.6 for a variety of bug fixes. (Mark Miller) + +* SOLR-3629: Typo in solr.xml persistence when overriding the solrconfig.xml + file name using the "config" attribute prevented the override file from being + used. (Ryan Zezeski, hossman) + +* SOLR-3642: Correct broken check for multivalued fields in stats.facet + (Yandong Yao, hossman) + +* SOLR-3660: Velocity: Link to admin page broken (janhoy) + +* SOLR-3658: Adding thousands of docs with one UpdateProcessorChain instance can briefly create + spikes of threads in the thousands. (yonik, Mark Miller) + +* SOLR-3656: A core reload now always uses the same dataDir. (Mark Miller, yonik) + +* SOLR-3662: Core reload bugs: a reload always obtained a non-NRT searcher, which + could go back in time with respect to the previous core's NRT searcher. Versioning + did not work correctly across a core reload, and update handler synchronization + was changed to synchronize on core state since more than on update handler + can coexist for a single index during a reload. (yonik) + +* SOLR-3663: There are a couple of bugs in the sync process when a leader goes down and a + new leader is elected. (Mark Miller) + +* SOLR-3623: Fixed inconsistent treatment of third-party dependencies for + solr contribs analysis-extras & uima (hossman) + +* SOLR-3652: Fixed range faceting to error instead of looping infinitely + when 'gap' is zero -- or effectively zero due to floating point arithmetic + underflow. (hossman) + +* SOLR-3648: Fixed VelocityResponseWriter template loading in SolrCloud mode. + For the example configuration, this means /browse now works with SolrCloud. + (janhoy, ehatcher) + +* SOLR-3677: Fixed misleading error message in web ui to distinguish between + no SolrCores loaded vs. no /admin/ handler available. + (hossman, steffkes) + +* SOLR-3428: SolrCmdDistributor flushAdds/flushDeletes can cause repeated + adds/deletes to be sent (Mark Miller, Per Steffensen) + +* SOLR-3647: DistributedQueue should use our Solr zk client rather than the std zk + client. ZooKeeper expiration can be permanent otherwise. (Mark Miller) + +Other Changes +---------------------- + +* SOLR-3524: Make discarding punctuation configurable in JapaneseTokenizerFactory. + The default is to discard punctuation, but this is overridable as an expert option. + (Kazuaki Hiraga, Jun Ohtani via Christian Moen) + +* SOLR-1770: Move the default core instance directory into a collection1 folder. + (Mark Miller) + +* SOLR-3355: Add shard and collection to SolrCore statistics. (Michael Garski, Mark Miller) + +* SOLR-3575: solr.xml should default to persist=true (Mark Miller) + +* SOLR-3563: Unloading all cores in a SolrCloud collection will now cause the removal of + that collection's meta data from ZooKeeper. (Mark Miller, Per Steffensen) + +* SOLR-3599: Add zkClientTimeout to solr.xml so that it's obvious how to change it and so + that you can change it with a system property. (Mark Miller) + +* SOLR-3609: Change Solr's expanded webapp directory to be at a consistent path called + solr-webapp rather than a temporary directory. (Mark Miller) + +* SOLR-3600: Raise the default zkClientTimeout from 10 seconds to 15 seconds. (Mark Miller) + +* SOLR-3215: Clone SolrInputDocument when distrib indexing so that update processors after + the distrib update process do not process the document twice. (Mark Miller) + +* SOLR-3683: Improved error handling if an <analyzer> contains both an + explicit class attribute, as well as nested factories. (hossman) + +* SOLR-3682: Fail to parse schema.xml if uniqueKeyField is multivalued (hossman) + +* SOLR-2115: DIH no longer requires the "config" parameter to be specified in solrconfig.xml. + Instead, the configuration is loaded and parsed with every import. This allows the use of + a different configuration with each import, and makes correcting configuration errors simpler. + Also, the configuration itself can be passed using the "dataConfig" parameter rather than + using a file (this previously worked in debug mode only). When configuration errors are + encountered, the error message is returned in XML format. (James Dyer) + +* SOLR-3439: Make SolrCell easier to use out of the box. Also improves "/browse" to display + rich-text documents correctly, along with facets for author and content_type. + With the new "content" field, highlighting of body is supported. See also SOLR-3672 for + easier posting of a whole directory structure. (Jack Krupansky, janhoy) + +* SOLR-3579: SolrCloud view should default to the graph view rather than tree view. + (steffkes, Mark Miller) + +================== 4.0.0-ALPHA ================== +More information about this release, including any errata related to the +release notes, upgrade instructions, or other changes may be found online at: + https://wiki.apache.org/solr/Solr4.0 + + +Versions of Major Components +--------------------- +Apache Tika 1.1 +Carrot2 3.5.0 +Velocity 1.6.4 and Velocity Tools 2.0 +Apache UIMA 2.3.1 +Apache ZooKeeper 3.3.4 + + +Upgrading from Solr 3.6-dev +---------------------- + +* The Lucene index format has changed and as a result, once you upgrade, + previous versions of Solr will no longer be able to read your indices. + In a master/slave configuration, all searchers/slaves should be upgraded + before the master. If the master were to be updated first, the older + searchers would not be able to read the new index format. + +* Setting abortOnConfigurationError=false is no longer supported + (since it has never worked properly). Solr will now warn you if + you attempt to set this configuration option at all. (see SOLR-1846) + +* The default logic for the 'mm' param of the 'dismax' QParser has + been changed. If no 'mm' param is specified (either in the query, + or as a default in solrconfig.xml) then the effective value of the + 'q.op' param (either in the query or as a default in solrconfig.xml + or from the 'defaultOperator' option in schema.xml) is used to + influence the behavior. If q.op is effectively "AND" then mm=100%. + If q.op is effectively "OR" then mm=0%. Users who wish to force the + legacy behavior should set a default value for the 'mm' param in + their solrconfig.xml file. + +* The VelocityResponseWriter is no longer built into the core. Its JAR and + dependencies now need to be added (via <lib> or solr/home lib inclusion), + and it needs to be registered in solrconfig.xml like this: + <queryResponseWriter name="velocity" class="solr.VelocityResponseWriter"/> + +* The update request parameter to choose Update Request Processor Chain is + renamed from "update.processor" to "update.chain". The old parameter was + deprecated but still working since Solr3.2, but is now removed + entirely. + +* The <indexDefaults> and <mainIndex> sections of solrconfig.xml are discontinued + and replaced with the <indexConfig> section. There are also better defaults. + When migrating, if you don't know what your old settings mean, simply delete + both <indexDefaults> and <mainIndex> sections. If you have customizations, + put them in <indexConfig> section - with same syntax as before. + +* Two of the SolrServer subclasses in SolrJ were renamed/replaced. + CommonsHttpSolrServer is now HttpSolrServer, and + StreamingUpdateSolrServer is now ConcurrentUpdateSolrServer. + +* The PingRequestHandler no longer looks for a <healthcheck/> option in the + (legacy) <admin> section of solrconfig.xml. Users who wish to take + advantage of this feature should configure a "healthcheckFile" init param + directly on the PingRequestHandler. As part of this change, relative file + paths have been fixed to be resolved against the data dir. See the example + solrconfig.xml and SOLR-1258 for more details. + +* Due to low level changes to support SolrCloud, the uniqueKey field can no + longer be populated via <copyField/> or <field default=...> in the + schema.xml. Users wishing to have Solr automatically generate a uniqueKey + value when adding documents should instead use an instance of + solr.UUIDUpdateProcessorFactory in their update processor chain. See + SOLR-2796 for more details. + + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-3272: Solr filter factory for MorfologikFilter (Polish lemmatisation). + (Rafał Kuć via Dawid Weiss, Steven Rowe, Uwe Schindler). + +* SOLR-571: The autowarmCount for LRUCaches (LRUCache and FastLRUCache) now + supports "percentages" which get evaluated relative the current size of + the cache when warming happens. + (Tomás Fernández Löbbe and hossman) + +* SOLR-1932: New relevancy function queries: termfreq, tf, docfreq, idf + norm, maxdoc, numdocs. (yonik) + +* SOLR-1665: Add debug component options for timings, results and query info only (gsingers, hossman, yonik) + +* SOLR-2112: Solrj API now supports streaming results. (ryan) + +* SOLR-792: Adding PivotFacetComponent for Hierarchical faceting + (ehatcher, Jeremy Hinegardner, Thibaut Lassalle, ryan) + +* LUCENE-2507, SOLR-2571, SOLR-2576: Added DirectSolrSpellChecker, which uses Lucene's + DirectSpellChecker to retrieve correction candidates directly from the term dictionary using + levenshtein automata. (James Dyer, rmuir) + +* SOLR-1873, SOLR-2358: SolrCloud - added shared/central config and core/shard management via zookeeper, + built-in load balancing, and distributed indexing. + (Jamie Johnson, Sami Siren, Ted Dunning, yonik, Mark Miller) + Additional Work: + - SOLR-2324: SolrCloud solr.xml parameters are not persisted by CoreContainer. + (Massimo Schiavon, Mark Miller) + - SOLR-2287: Allow users to query by multiple, compatible collections with SolrCloud. + (Soheb Mahmood, Alex Cowell, Mark Miller) + - SOLR-2622: ShowFileRequestHandler does not work in SolrCloud mode. + (Stefan Matheis, Mark Miller) + - SOLR-3108: Error in SolrCloud's replica lookup code when replica's are hosted in same Solr instance. + (Bruno Dumon, Sami Siren, Mark Miller) + - SOLR-3080: Remove shard info from zookeeper when SolrCore is explicitly unloaded. + (yonik, Mark Miller, siren) + - SOLR-3437: Recovery issues a spurious commit to the cluster. (Trym R. Møller via Mark Miller) + - SOLR-2822: Skip update processors already run on other nodes (hossman) + +* SOLR-1566: Transforming documents in the ResponseWriters. This will allow + for more complex results in responses and open the door for function queries + as results. + (ryan with patches from grant, noble, cmale, yonik, Jan Høydahl, + Arul Kalaipandian, Luca Cavanna, hossman) + - SOLR-2037: Thanks to SOLR-1566, documents boosted by the QueryElevationComponent + can be marked as boosted. (gsingers, ryan, yonik) + +* SOLR-2396: Add CollationField, which is much more efficient than + the Solr 3.x CollationKeyFilterFactory, and also supports + Locale-sensitive range queries. (rmuir) + +* SOLR-2338: Add support for using <similarity/> in a schema's fieldType, + for customizing scoring on a per-field basis. (hossman, yonik, rmuir) + +* SOLR-2335: New 'field("...")' function syntax for referring to complex + field names (containing whitespace or special characters) in functions. + +* SOLR-2383: /browse improvements: generalize range and date facet display + (Jan Høydahl via yonik) + +* SOLR-2272: Pseudo-join queries / filters. Examples: + - To restrict to the set of parents with at least one blue-eyed child: + fq={!join from=parent to=name}eyes:blue + - To restrict to the set of children with at least one blue-eyed parent: + fq={!join from=name to=parent}eyes:blue + (yonik) + +* SOLR-1942: Added the ability to select postings format per fieldType in schema.xml + as well as support custom Codecs in solrconfig.xml. + (simonw via rmuir) + +* SOLR-2136: Boolean type added to function queries, along with + new functions exists(), if(), and(), or(), xor(), not(), def(), + and true and false constants. (yonik) + +* SOLR-2491: Add support for using spellcheck collation in conjunction + with grouping. Note that the number of hits returned for collations + is the number of ungrouped hits. (James Dyer via rmuir) + +* SOLR-1298: Return FunctionQuery as pseudo field. The solr 'fl' param + now supports functions. For example: fl=id,sum(x,y) -- NOTE: only + functions with fast random access are recommended. (yonik, ryan) + +* SOLR-705: Optionally return shard info with each document in distributed + search. Use fl=id,[shard] to return the shard url. (ryan) + +* SOLR-2417: Add explain info directly to return documents using + ?fl=id,[explain] (ryan) + +* SOLR-2533: Converted ValueSource.ValueSourceSortField over to new rewriteable Lucene + SortFields. ValueSourceSortField instances must be rewritten before they can be used. + This is done by SolrIndexSearcher when necessary. (Chris Male). + +* SOLR-2193, SOLR-2565: You may now specify a 'soft' commit when committing. This will + use Lucene's NRT feature to avoid guaranteeing documents are on stable storage in exchange + for faster reopen times. There is also a new 'soft' autocommit tracker that can be + configured. (Mark Miller, Robert Muir) + +* SOLR-2399: Updated Solr Admin interface. New look and feel with per core administration + and many new options. (Stefan Matheis via ryan) + +* SOLR-1032: CSV handler now supports "literal.field_name=value" parameters. + (Simon Rosenthal, ehatcher) + +* SOLR-2656: realtime-get, efficiently retrieves the latest stored fields for specified + documents, even if they are not yet searchable (i.e. without reopening a searcher) + (yonik) + +* SOLR-2703: Added support for Lucene's "surround" query parser. (Simon Rosenthal, ehatcher) + +* SOLR-2754: Added factories for several ranking algorithms: + - BM25SimilarityFactory: Okapi BM25 + - DFRSimilarityFactory: Divergence from Randomness models + - IBSimilarityFactory: Information-based models + - LMDirichletSimilarity: LM with Dirichlet smoothing + - LMJelinekMercerSimilarity: LM with Jelinek-Mercer smoothing + (David Mark Nemeskey, Robert Muir) + +* SOLR-2134 Trie* fields should support sortMissingLast=true, and deprecate Sortable* Field Types + (Ryan McKinley, Mike McCandless, Uwe Schindler, Erick Erickson) + +* SOLR-2438 added MultiTermAwareComponent to the various classes to allow automatic lowercasing + for multiterm queries (wildcards, regex, prefix, range, etc). You can now optionally specify a + "multiterm" analyzer in our schema.xml, but Solr should "do the right thing" if you don't + specify <analyzer type="multiterm"> (Pete Sturge Erick Erickson, Mentoring from Seeley and Muir) + +* SOLR-2481: Add support for commitWithin in DataImportHandler (Sami Siren via yonik) + +* SOLR-2992: Add support for IndexWriter.prepareCommit() via prepareCommit=true + on update URLs. (yonik) + +* SOLR-2906: Added LFU cache options to Solr. (Shawn Heisey via Erick Erickson) + +* SOLR-3069: Ability to add openSearcher=false to not open a searcher when doing + a hard commit. commitWithin now only invokes a softCommit. (yonik) + +* SOLR-2802: New FieldMutatingUpdateProcessor and Factory to simplify the + development of UpdateProcessors that modify field values of documents as + they are indexed. Also includes several useful new implementations: + - RemoveBlankFieldUpdateProcessorFactory + - TrimFieldUpdateProcessorFactory + - HTMLStripFieldUpdateProcessorFactory + - RegexReplaceProcessorFactory + - FieldLengthUpdateProcessorFactory + - ConcatFieldUpdateProcessorFactory + - FirstFieldValueUpdateProcessorFactory + - LastFieldValueUpdateProcessorFactory + - MinFieldValueUpdateProcessorFactory + - MaxFieldValueUpdateProcessorFactory + - TruncateFieldUpdateProcessorFactory + - IgnoreFieldUpdateProcessorFactory + (hossman, janhoy) + +* SOLR-3120: Optional post filtering for spatial queries bbox and geofilt + for LatLonType. (yonik) + +* SOLR-2459: Expose LogLevel selection with a RequestHandler rather then servlet + (Stefan Matheis, Upayavira, ryan) + +* SOLR-3134: Include shard info in distributed response when shards.info=true + (Russell Black, ryan) + +* SOLR-2898: Support grouped faceting. (Martijn van Groningen) + Additional Work: + - SOLR-3406: Extended grouped faceting support to facet.query and facet.range parameters. + (David Boychuck, Martijn van Groningen) + +* SOLR-2949: QueryElevationComponent is now supported with distributed search. + (Mark Miller, yonik) + +* SOLR-3221: Added the ability to directly configure aspects of the concurrency + and thread-pooling used within distributed search in solr. This allows for finer + grained controlled and can be tuned by end users to target their own specific + requirements. This builds on the work of the HttpCommComponent and uses the same configuration + block to configure the thread pool. The default configuration has + the same behaviour as solr 3.5, favouring throughput over latency. More + information can be found on the wiki (http://wiki.apache.org/solr/SolrConfigXml) (Greg Bowyer) + +* SOLR-3278: Negative boost support to the Extended Dismax Query Parser Boost Query (bq). + (James Dyer) + +* SOLR-3255: OpenExchangeRates.Org Exchange Rate Provider for CurrencyField (janhoy) + +* SOLR-3358: Logging events are captured and available from the /admin/logging + request handler. (ryan) + +* SOLR-1535: PreAnalyzedField type provides a functionality to index (and optionally store) + field content that was already processed and split into tokens using some external processing + chain. Serialization format is pluggable, and defaults to JSON. (ab) + +* SOLR-3363: Consolidated Exceptions in Analysis Factories so they only throw + InitializationExceptions (Chris Male) + +* SOLR-2690: New support for a "TZ" request param which overrides the TimeZone + used when rounding Dates in DateMath expressions for the entire request + (all date range queries and date faceting is affected). The default TZ + is still UTC. (David Schlotfeldt, hossman) + +* SOLR-3402: Analysis Factories are now configured with their Lucene Version + throw setLuceneMatchVersion, rather than through the Map passed to init. + Parsing and simple error checking for the Version is now done inside + the code that creates the Analysis Factories. (Chris Male) + +* SOLR-3178: Optimistic locking. If a _version_ is provided with an update + that does not match the version in the index, an HTTP 409 error (Conflict) + will result. (Per Steffensen, yonik) + +* SOLR-139: Updateable documents. JSON Example: + {"id":"mydoc", "f1":{"set":10}, "f2":{"add":20}} will result in field "f1" + being set to 10, "f2" having an additional value of 20 added, and all + other existing fields unchanged. All source fields must be stored for + this feature to work correctly. (Ryan McKinley, Erik Hatcher, yonik) + +* SOLR-2857: Support XML,CSV,JSON, and javabin in a single RequestHandler and + choose the correct ContentStreamLoader based on Content-Type header. This + also deprecates the existing [Xml,JSON,CSV,Binary,Xslt]UpdateRequestHandler. + (ryan) + +* SOLR-2585: Context-Sensitive Spelling Suggestions & Collations. This adds support + for the "spellcheck.alternativeTermCount" & "spellcheck.maxResultsForSuggest" + parameters, letting users receive suggestions even when all the queried terms + exist in the dictionary. This differs from "spellcheck.onlyMorePopular" in + that the suggestions need not consist entirely of terms with a greater document + frequency than the queried terms. (James Dyer) + +* SOLR-2058: Edismax query parser to allow "phrase slop" to be specified per-field + on the pf/pf2/pf3 parameters using optional "FieldName~slop^boost" syntax. The + prior "FieldName^boost" syntax is still accepted. In such cases the value on the + "ps" parameter serves as the default slop. (Ron Mayer via James Dyer) + +* SOLR-3495: New UpdateProcessors have been added to create default values for + configured fields. These works similarly to the <field default="..."/> + option in schema.xml, but are applied in the UpdateProcessorChain, so they + may be used prior to other UpdateProcessors, or to generate a uniqueKey field + value when using the DistributedUpdateProcessor (ie: SolrCloud) + TimestampUpdateProcessorFactory + UUIDUpdateProcessorFactory + DefaultValueUpdateProcessorFactory + (hossman) + +* SOLR-2993: Add WordBreakSolrSpellChecker to offer suggestions by combining adjacent + query terms and/or breaking terms into multiple words. This spellchecker can be + configured with a traditional checker (ie: DirectSolrSpellChecker). The results + are combined and collations can contain a mix of corrections from both spellcheckers. + (James Dyer) + +* SOLR-3508: Simplify JSON update format for deletes as well as allow + version specification for optimistic locking. Examples: + - {"delete":"myid"} + - {"delete":["id1","id2","id3"]} + - {"delete":{"id":"myid", "_version_":123456789}} + (yonik) + +* SOLR-3211: Allow parameter overrides in conjunction with "spellcheck.maxCollationTries". + To do so, use parameters starting with "spellcheck.collateParam." For instance, to + override the "mm" parameter, specify "spellcheck.collateParam.mm". This is helpful + in cases where testing spellcheck collations for result counts should use different + parameters from the main query (James Dyer) + +* SOLR-2599: CloneFieldUpdateProcessorFactory provides similar functionality + to schema.xml's <copyField/> declaration but as an update processor that can + be combined with other processors in any order. (Jan Høydahl & hossman) + +* SOLR-3351: eDismax: ps2 and ps3 params (janhoy) + +* SOLR-3542: Add WeightedFragListBuilder for FVH and set it to default fragListBuilder + in example solrconfig.xml. (Sebastian Lutze, koji) + +* SOLR-2396: Add ICUCollationField to contrib/analysis-extras, which is much + more efficient than the Solr 3.x ICUCollationKeyFilterFactory, and also + supports Locale-sensitive range queries. (rmuir) + + +Optimizations +---------------------- + +* SOLR-1875: Per-segment field faceting for single valued string fields. + Enable with facet.method=fcs, control the number of threads used with + the "threads" local param on the facet.field param. This algorithm will + only be faster in the presence of rapid index changes. (yonik) + +* SOLR-1904: When facet.enum.cache.minDf > 0 and the base doc set is a + SortedIntSet, convert to HashDocSet for better performance. (yonik) + +* SOLR-2092: Speed up single-valued and multi-valued "fc" faceting. Typical + improvement is 5%, but can be much greater (up to 10x faster) when facet.offset + is very large (deep paging). (yonik) + +* SOLR-2193, SOLR-2565: The default Solr update handler has been improved so + that it uses fewer locks, keeps the IndexWriter open rather than closing it + on each commit (ie commits no longer wait for background merges to complete), + works with SolrCore to provide faster 'soft' commits, and has an improved API + that requires less instanceof special casing. (Mark Miller, Robert Muir) + Additional Work: + - SOLR-2697: commit and autocommit operations don't reset + DirectUpdateHandler2.numDocsPending stats attribute. + (Alexey Serba, Mark Miller) + +* SOLR-2950: The QueryElevationComponent now avoids using the FieldCache and looking up + every document id (gsingers, yonik) + +Bug Fixes +---------------------- +* SOLR-3139: Make ConcurrentUpdateSolrServer send UpdateRequest.getParams() + as HTTP request params (siren) + +* SOLR-3165: Cannot use DIH in Solrcloud + Zookeeper (Alexey Serba, + Mark Miller, siren) + +* SOLR-3068: Occasional NPE in ThreadDumpHandler (siren) + +* SOLR-2762: FSTLookup could return duplicate results or one results less + than requested. (David Smiley, Dawid Weiss) + +* SOLR-2741: Bugs in facet range display in trunk (janhoy) + +* SOLR-1908: Fixed SignatureUpdateProcessor to fail to initialize on + invalid config. Specifically: a signatureField that does not exist, + or overwriteDupes=true with a signatureField that is not indexed. + (hossman) + +* SOLR-1824: IndexSchema will now fail to initialize if there is a + problem initializing one of the fields or field types. (hossman) + +* SOLR-1928: TermsComponent didn't correctly break ties for non-text + fields sorted by count. (yonik) + +* SOLR-2107: MoreLikeThisHandler doesn't work with alternate qparsers. (yonik) + +* SOLR-2108: Fixed false positives when using wildcard queries on fields with reversed + wildcard support. For example, a query of *zemog* would match documents that contain + 'gomez'. (Landon Kuhn via Robert Muir) + +* SOLR-1962: SolrCore#initIndex should not use a mix of indexPath and newIndexPath (Mark Miller) + +* SOLR-2275: fix DisMax 'mm' parsing to be tolerant of whitespace + (Erick Erickson via hossman) + +* SOLR-2193, SOLR-2565, SOLR-2651: SolrCores now properly share IndexWriters across SolrCore reloads. + (Mark Miller, Robert Muir) + Additional Work: + - SOLR-2705: On reload, IndexWriterProvider holds onto the initial SolrCore it was created with. + (Yury Kats, Mark Miller) + +* SOLR-2682: Remove addException() in SimpleFacet. FacetComponent no longer catches and embeds + exceptions occurred during facet processing, it throws HTTP 400 or 500 exceptions instead. (koji) + +* SOLR-2654: Directorys used by a SolrCore are now closed when they are no longer used. + (Mark Miller) + +* SOLR-2854: Now load URL content stream data (via stream.url) when called for during request handling, + rather than loading URL content streams automatically regardless of use. + (David Smiley and Ryan McKinley via ehatcher) + +* SOLR-2829: Fix problem with false-positives due to incorrect + equals methods. (Yonik Seeley, Hossman, Erick Erickson. + Marc Tinnemeyer caught the bug) + +* SOLR-2848: Removed 'instanceof AbstractLuceneSpellChecker' hacks from distributed spellchecking code, + and added a merge() method to SolrSpellChecker instead. Previously if you extended SolrSpellChecker + your spellchecker would not work in distributed fashion. (James Dyer via rmuir) + +* SOLR-2509: StringIndexOutOfBoundsException in the spellchecker collate when the term contains + a hyphen. (Thomas Gambier caught the bug, Steffen Godskesen did the patch, via Erick Erickson) + +* SOLR-1730: Made it clearer when a core failed to load as well as better logging when the + QueryElevationComponent fails to properly initialize (gsingers) + +* SOLR-1520: QueryElevationComponent now supports non-string ids (gsingers) + +* SOLR-3037: When using binary format in solrj the codec screws up parameters + (Sami Siren, Jörg Maier via yonik) + +* SOLR-3062: A join in the main query was not respecting any filters pushed + down to it via acceptDocs since LUCENE-1536. (Mike Hugo, yonik) + +* SOLR-3214: If you use multiple fl entries rather than a comma separated list, all but the first + entry can be ignored if you are using distributed search. (Tomás Fernández Löbbe via Mark Miller) + +* SOLR-3352: eDismax: pf2 should kick in for a query with 2 terms (janhoy) + +* SOLR-3361: ReplicationHandler "maxNumberOfBackups" doesn't work if backups are triggered on commit + (James Dyer, Tomás Fernández Löbbe) + +* SOLR-2605: fixed tracking of the 'defaultCoreName' in CoreContainer so that + CoreAdminHandler could return consistent information regardless of whether + there is a a default core name or not. (steffkes, hossman) + +* SOLR-3370: fixed CSVResponseWriter to respect globs in the 'fl' param + (Keith Fligg via hossman) + +* SOLR-3436: Group count incorrect when not all shards are queried in the second + pass. (Francois Perron, Martijn van Groningen) + +* SOLR-3454: Exception when using result grouping with main=true and using + wt=javabin. (Ludovic Boutros, Martijn van Groningen) + +* SOLR-3446: Better errors when PatternTokenizerFactory is configured with + an invalid pattern, and include the 'name' whenever possible in plugin init + error messages. (hossman) + +* LUCENE-4075: Cleaner path usage in TestXPathEntityProcessor + (Greg Bowyer via hossman) + +* SOLR-2923: IllegalArgumentException when using useFilterForSortedQuery on an + empty index. (Adrien Grand via Mark Miller) + +* SOLR-2352: Fixed TermVectorComponent so that it will not fail if the fl + param contains globs or psuedo-fields (hossman) + +* SOLR-3541: add missing solrj dependencies to binary packages. + (Thijs Vonk via siren) + +* SOLR-3522: fixed parsing of the 'literal()' function (hossman) + +* SOLR-3548: Fixed a bug in the cachability of queries using the {!join} + parser or the strdist() function, as well as some minor improvements to + the hashCode implementation of {!bbox} and {!geofilt} queries. + (hossman) + +* SOLR-3470: contrib/clustering: custom Carrot2 tokenizer and stemmer factories + are respected now (Stanislaw Osinski, Dawid Weiss) + +* SOLR-3430: Added a new DIH test against a real SQL database. Fixed problems + revealed by this new test related to the expanded cache support added to + 3.6/SOLR-2382 (James Dyer) + +* SOLR-1958: When using the MailEntityProcessor, import would fail if + fetchMailsSince was not specified. (Max Lynch via James Dyer) + +* SOLR-4289: Admin UI - JVM memory bar - dark grey "used" width is too small + (steffkes, elyograg) + +Other Changes +---------------------- + +* SOLR-1846: Eliminate support for the abortOnConfigurationError + option. It has never worked very well, and in recent versions of + Solr hasn't worked at all. (hossman) + +* SOLR-1889: The default logic for the 'mm' param of DismaxQParser and + ExtendedDismaxQParser has been changed to be determined based on the + effective value of the 'q.op' param (hossman) + +* SOLR-1946: Misc improvements to the SystemInfoHandler: /admin/system + (hossman) + +* SOLR-2289: Tweak spatial coords for example docs so they are a bit + more spread out (Erick Erickson via hossman) + +* SOLR-2288: Small tweaks to eliminate compiler warnings. primarily + using Generics where applicable in method/object declarations, and + adding @SuppressWarnings("unchecked") when appropriate (hossman) + +* SOLR-2375: Suggester Lookup implementations now store trie data + and load it back on init. This means that large tries don't have to be + rebuilt on every commit or core reload. (ab) + +* SOLR-2413: Support for returning multi-valued fields w/o <arr> tag + in the XMLResponseWriter was removed. XMLResponseWriter only + no longer work with values less then 2.2 (ryan) + +* SOLR-2423: FieldType argument changed from String to Object + Conversion from SolrInputDocument > Object > Fieldable is now managed + by FieldType rather then DocumentBuilder. (ryan) + +* SOLR-2461: QuerySenderListener and AbstractSolrEventListener are + now public (hossman) + +* LUCENE-2995: Moved some spellchecker and suggest APIs to modules/suggest: + HighFrequencyDictionary, SortedIterator, TermFreqIterator, and the + suggester APIs and implementations. (rmuir) + +* SOLR-2576: Remove deprecated SpellingResult.add(Token, int). + (James Dyer via rmuir) + +* LUCENE-3232: Moved MutableValue classes to new 'common' module. (Chris Male) + +* LUCENE-2883: FunctionQuery, DocValues (and its impls), ValueSource (and its + impls) and BoostedQuery have been consolidated into the queries module. They + can now be found at o.a.l.queries.function. + +* SOLR-2027: FacetField.getValues() now returns an empty list if there are no + values, instead of null (Chris Male) + +* SOLR-1825: SolrQuery.addFacetQuery now enables facets automatically, like + addFacetField (Chris Male) + +* SOLR-2663: FieldTypePluginLoader has been refactored out of IndexSchema + and made public. (hossman) + +* SOLR-2331,SOLR-2691: Refactor CoreContainer's SolrXML serialization code and improve testing + (Yury Kats, hossman, Mark Miller) + +* SOLR-2698: Enhance CoreAdmin STATUS command to return index size. + (Yury Kats, hossman, Mark Miller) + +* SOLR-2654: The same Directory instance is now always used across a SolrCore so that + it's easier to add other DirectoryFactory's without static caching hacks. + (Mark Miller) + +* LUCENE-3286: 'luke' ant target has been disabled due to incompatibilities with XML + queryparser location (Chris Male) + +* SOLR-1897: The data dir from the core descriptor should override the data dir from + the solrconfig.xml rather than the other way round. (Mark Miller) + +* SOLR-2756: Maven configuration: Excluded transitive stax:stax-api dependency + from org.codehaus.woodstox:wstx-asl dependency. (David Smiley via Steve Rowe) + +* SOLR-2588: Moved VelocityResponseWriter back to contrib module in order to + remove it as a mandatory core dependency. (ehatcher) + +* SOLR-2862: More explicit lexical resources location logged if Carrot2 clustering + extension is used. Fixed solr. impl. of IResource and IResourceLookup. (Dawid Weiss) + +* SOLR-1123: Changed JSONResponseWriter to now use application/json as its Content-Type + by default. However the Content-Type can be overwritten and is set to text/plain in + the example configuration. (Uri Boness, Chris Male) + +* SOLR-2607: Removed deprecated client/ruby directory, which included solr-ruby and flare. + (ehatcher) + +* SOLR-3032: logOnce from SolrException logOnce and all the supporting + structure is gone. abortOnConfigurationError is also gone as it is no longer referenced. + Errors should be caught and logged at the top-most level or logged and NOT propagated up the + chain. (Erick Erickson) + +* SOLR-2105: Remove support for deprecated "update.processor" (since 3.2), in favor of + "update.chain" (janhoy) + +* SOLR-3005: Default QueryResponseWriters are now initialized via init() with an empty + NamedList. (Gasol Wu, Chris Male) + +* SOLR-2607: Removed obsolete client/ folder (ehatcher, Eric Pugh, janhoy) + +* SOLR-3202, SOLR-3244: Dropping Support for JSP. New Admin UI is all client side + (ryan, Aliaksandr Zhuhrou, Uwe Schindler) + +* SOLR-3159: Upgrade example and tests to run with Jetty 8 (ryan) + +* SOLR-3254: Upgrade Solr to Tika 1.1 (janhoy) + +* SOLR-3329: Dropped getSourceID() from SolrInfoMBean and using + getClass().getPackage().getSpecificationVersion() for Version. (ryan) + +* SOLR-3302: Upgraded SLF4j to version 1.6.4 (hossman) + +* SOLR-3322: Add more context to IndexReaderFactory.newReader (ab) + +* SOLR-3343: Moved FastWriter, FileUtils, RegexFileFilter, RTimer and SystemIdResolver + from org.apache.solr.common to org.apache.solr.util (Chris Male) + +* SOLR-3357: ResourceLoader.newInstance now accepts a Class representation of the expected + instance type (Chris Male) + +* SOLR-3388: HTTP caching is now disabled by default for RequestUpdateHandlers. (ryan) + +* SOLR-3309: web.xml now specifies metadata-complete=true (which requires + Servlet 2.5) to prevent servlet containers from scanning class annotations + on startup. This allows for faster startup times on some servlet containers. + (Bill Bell, hossman) + +* SOLR-1893: Refactored some common code from LRUCache and FastLRUCache into + SolrCacheBase (Tomás Fernández Löbbe via hossman) + +* SOLR-3403: Deprecated Analysis Factories now log their own deprecation messages. + No logging support is provided by Factory parent classes. (Chris Male) + +* SOLR-1258: PingRequestHandler is now directly configured with a + "healthcheckFile" instead of looking for the legacy + <admin><healthcheck/></admin> syntax. Filenames specified as relative + paths have been fixed so that they are resolved against the data dir + instead of the CWD of the java process. (hossman) + +* SOLR-3083: JMX beans now report Numbers as numeric values rather then String + (Tagged Siteops, Greg Bowyer via ryan) + +* SOLR-2796: Due to low level changes to support SolrCloud, the uniqueKey + field can no longer be populated via <copyField/> or <field default=...> + in the schema.xml. + +* SOLR-3534: The Dismax and eDismax query parsers will fall back on the 'df' parameter + when 'qf' is absent. And if neither is present nor the schema default search field + then an exception will be thrown now. (dsmiley) + +* SOLR-3262: The "threads" feature of DIH is removed (deprecated in Solr 3.6) + (James Dyer) + +* SOLR-3422: Refactored DIH internal data classes. All entities in + data-config.xml must have a name (James Dyer) + +Documentation +---------------------- + +* SOLR-2232: Improved README info on solr.solr.home in examples + (Eric Pugh and hossman) + +================== 3.6.2 ================== + +Bug Fixes +---------------------- +* SOLR-3790: ConcurrentModificationException could be thrown when using hl.fl=*. + (yonik, koji) + +* SOLR-3589: Edismax parser does not honor mm parameter if analyzer splits a token. + (Tom Burton-West, Robert Muir) + +================== 3.6.1 ================== +More information about this release, including any errata related to the +release notes, upgrade instructions, or other changes may be found online at: + https://wiki.apache.org/solr/Solr3.6.1 + +Bug Fixes + +* LUCENE-3969: Throw IAE on bad arguments that could cause confusing errors in + PatternTokenizer. CommonGrams populates PositionLengthAttribute correctly. + (Uwe Schindler, Mike McCandless, Robert Muir) + +* SOLR-3361: ReplicationHandler "maxNumberOfBackups" doesn't work if backups are triggered on commit + (James Dyer, Tomás Fernández Löbbe) + +* SOLR-3375: Fix charset problems with HttpSolrServer (Roger Håkansson, yonik, siren) + +* SOLR-3436: Group count incorrect when not all shards are queried in the second + pass. (Francois Perron, Martijn van Groningen) + +* SOLR-3454: Exception when using result grouping with main=true and using + wt=javabin. (Ludovic Boutros, Martijn van Groningen) + +* SOLR-3489: Config file replication less error prone (Jochen Just via janhoy) + +* SOLR-3477: SOLR does not start up when no cores are defined (Tomás Fernández Löbbe via tommaso) + +* SOLR-3470: contrib/clustering: custom Carrot2 tokenizer and stemmer factories + are respected now (Stanislaw Osinski, Dawid Weiss) + +* SOLR-3360: More DIH bug fixes for the deprecated "threads" parameter. + (Mikhail Khludnev, Claudio R, via James Dyer) + +* SOLR-3430: Added a new DIH test against a real SQL database. Fixed problems + revealed by this new test related to the expanded cache support added to + 3.6/SOLR-2382 (James Dyer) + +* SOLR-3336: SolrEntityProcessor substitutes most variables at query time. + (Michael Kroh, Lance Norskog, via Martijn van Groningen) + + +================== 3.6.0 ================== +More information about this release, including any errata related to the +release notes, upgrade instructions, or other changes may be found online at: + https://wiki.apache.org/solr/Solr3.6 + +Upgrading from Solr 3.5 +---------------------- +* SOLR-2983: As a consequence of moving the code which sets a MergePolicy from SolrIndexWriter to SolrIndexConfig, + (custom) MergePolicies should now have an empty constructor; thus an IndexWriter should not be passed as constructor + parameter but instead set using the setIndexWriter() method. + +* As doGet() methods in SimplePostTool was changed to static, the client applications of this + class need to be recompiled. + +* In Solr version 3.5 and earlier, HTMLStripCharFilter had known bugs in the + character offsets it provided, triggering e.g. exceptions in highlighting. + HTMLStripCharFilter has been re-implemented, addressing this and other + issues. See the entry for LUCENE-3690 in the Bug Fixes section below for a + detailed list of changes. For people who depend on the behavior of + HTMLStripCharFilter in Solr version 3.5 and earlier: the old implementation + (bugs and all) is preserved as LegacyHTMLStripCharFilter. + +* As of Solr 3.6, the <indexDefaults> and <mainIndex> sections of solrconfig.xml are deprecated + and replaced with a new <indexConfig> section. Read more in SOLR-1052 below. + +* SOLR-3040: The DIH's admin UI (dataimport.jsp) now requires DIH request handlers to start with + a '/'. (dsmiley) + +* SOLR-3161: <requestDispatcher handleSelect="false"> is now the default. An existing config will + probably work as-is because handleSelect was explicitly enabled in default configs. HandleSelect + makes /select work as well as enables the 'qt' parameter. Instead, consider explicitly + configuring /select as is done in the example solrconfig.xml, and register your other search + handlers with a leading '/' which is a recommended practice. (David Smiley, Erik Hatcher) + +* SOLR-3161: Don't use the 'qt' parameter with a leading '/'. It probably won't work in 4.0 + and it's now limited in 3.6 to SearchHandler subclasses that aren't lazy-loaded. + +* SOLR-2724: Specifying <defaultSearchField> and <solrQueryParser defaultOperator="..."/> in + schema.xml is now considered deprecated. Instead you are encouraged to specify these via the "df" + and "q.op" parameters in your request handler definition. (David Smiley) + +* Bugs found and fixed in the SignatureUpdateProcessor that previously caused + some documents to produce the same signature even when the configured fields + contained distinct (non-String) values. Users of SignatureUpdateProcessor + are strongly advised that they should re-index as document signatures may + have now changed. (see SOLR-3200 & SOLR-3226 for details) + +New Features +---------------------- +* SOLR-2020: Add Java client that uses Apache Http Components http client (4.x). + (Chantal Ackermann, Ryan McKinley, Yonik Seeley, siren) + +* SOLR-2854: Now load URL content stream data (via stream.url) when called for during request handling, + rather than loading URL content streams automatically regardless of use. + (David Smiley and Ryan McKinley via ehatcher) + +* SOLR-2904: BinaryUpdateRequestHandler should be able to accept multiple update requests from + a stream (shalin) + +* SOLR-1565: StreamingUpdateSolrServer supports RequestWriter API and therefore, javabin update + format (shalin) + +* SOLR-2438 added MultiTermAwareComponent to the various classes to allow automatic lowercasing + for multiterm queries (wildcards, regex, prefix, range, etc). You can now optionally specify a + "multiterm" analyzer in our schema.xml, but Solr should "do the right thing" if you don't + specify <fieldType="multiterm"> (Pete Sturge Erick Erickson, Mentoring from Seeley and Muir) + +* SOLR-2919: Added support for localized range queries when the analysis chain uses + CollationKeyFilter or ICUCollationKeyFilter. (Michael Sokolov, rmuir) + +* SOLR-2982: Added BeiderMorseFilterFactory for Beider-Morse (BMPM) phonetic encoder. Upgrades + commons-codec to version 1.6 (Brooke Schreier Ganz, rmuir) + +* SOLR-1843: A new "rootName" attribute is now available when + configuring <jmx/> in solrconfig.xml. If this attribute is set, + Solr will use it as the root name for all MBeans Solr exposes via + JMX. The default root name is "solr" followed by the core name. + (Constantijn Visinescu, hossman) + +* SOLR-2906: Added LFU cache options to Solr. (Shawn Heisey via Erick Erickson) + +* SOLR-3036: Ability to specify overwrite=false on the URL for XML updates. + (Sami Siren via yonik) + +* SOLR-2603: Add the encoding function for alternate fields in highlighting. + (Massimo Schiavon, koji) + +* SOLR-1729: Evaluation of NOW for date math is done only once per request for + consistency, and is also propagated to shards in distributed search. + Adding a parameter NOW=<time_in_ms> to the request will override the + current time. (Peter Sturge, yonik, Simon Willnauer) + +* SOLR-1709: Distributed support for Date and Numeric Range Faceting + (Peter Sturge, David Smiley, hossman, Simon Willnauer) + +* SOLR-3054, LUCENE-3671: Add TypeTokenFilterFactory that creates TypeTokenFilter + that filters tokens based on their TypeAttribute. (Tommaso Teofili via + Uwe Schindler) + +* LUCENE-3305, SOLR-3056: Added Kuromoji morphological analyzer for Japanese. + See the 'text_ja' fieldtype in the example to get started. + (Christian Moen, Masaru Hasegawa via Robert Muir) + +* SOLR-1860: StopFilterFactory, CommonGramsFilterFactory, and + CommonGramsQueryFilterFactory can optionally read stopwords in Snowball + format (specify format="snowball"). (Robert Muir) + +* SOLR-3105: ElisionFilterFactory optionally allows the parameter + ignoreCase (default=false). (Robert Muir) + +* LUCENE-3714: Add WFSTLookupFactory, a suggester that uses a weighted FST + for more fine-grained suggestions. (Mike McCandless, Dawid Weiss, Robert Muir) + +* SOLR-3143: Add SuggestQueryConverter, a QueryConverter intended for + auto-suggesters. (Robert Muir) + +* SOLR-3033: ReplicationHandler's backup command now supports a 'maxNumberOfBackups' + init param that can be used to delete all but the most recent N backups. (Torsten Krah, James Dyer) + +* SOLR-2202: Currency FieldType, whith support for currencies and exchange rates + (Greg Fodor & Andrew Morrison via janhoy, rmuir, Uwe Schindler) + +* SOLR-3026: eDismax: Locking down which fields can be explicitly queried (user fields aka uf) + (janhoy, hossmann, Tomás Fernández Löbbe) + +* SOLR-2826: URLClassify Update Processor (janhoy) + +* SOLR-2764: Create a NorwegianLightStemmer and NorwegianMinimalStemmer (janhoy) + +* SOLR-3221: Added the ability to directly configure aspects of the concurrency + and thread-pooling used within distributed search in solr. This allows for finer + grained controlled and can be tuned by end users to target their own specific + requirements. This builds on the work of the HttpCommComponent and uses the same configuration + block to configure the thread pool. The default configuration has + the same behaviour as solr 3.5, favouring throughput over latency. More + information can be found on the wiki (http://wiki.apache.org/solr/SolrConfigXml) (Greg Bowyer) + +* SOLR-2001: The query component will substitute an empty query that matches + no documents if the query parser returns null. This also prevents an + exception from being thrown by the default parser if "q" is missing. (yonik) + - SOLR-435: if q is "" then it's also acceptable. (dsmiley, hoss) + +* SOLR-2919: Added parametric tailoring options to ICUCollationKeyFilterFactory. + These can be used to customize range query/sort behavior, for example to + support numeric collation, ignore punctuation/whitespace, ignore accents but + not case, control whether upper/lowercase values are sorted first, etc. (rmuir) + +* SOLR-2346: Add a chance to set content encoding explicitly via content type + of stream for extracting request handler. This is convenient when Tika's + auto detector cannot detect encoding, especially the text file is too short + to detect encoding. (koji) + +* SOLR-1499: Added SolrEntityProcessor that imports data from another Solr core + or instance based on a specified query. + (Lance Norskog, Erik Hatcher, Pulkit Singhal, Ahmet Arslan, Luca Cavanna, + Martijn van Groningen) + +* SOLR-3190: Minor improvements to SolrEntityProcessor. Add more consistency + between solr parameters and parameters used in SolrEntityProcessor and + ability to specify a custom HttpClient instance. + (Luca Cavanna via Martijn van Groningen) + +* SOLR-2382: Added pluggable cache support to DIH so that any Entity can be + made cache-able by adding the "cacheImpl" parameter. Include + "SortedMapBackedCache" to provide in-memory caching (as previously this was + the only option when using CachedSqlEntityProcessor). Users can provide + their own implementations of DIHCache for other caching strategies. + Deprecate CachedSqlEntityProcessor in favor of specifing "cacheImpl" with + SqlEntityProcessor. Make SolrWriter implement DIHWriter and allow the + possibility of pluggable Writers (DIH writing to something other than Solr). + (James Dyer, Noble Paul) + + +Optimizations +---------------------- +* SOLR-1931: Speedup for LukeRequestHandler and admin/schema browser. New parameter + reportDocCount defaults to 'false'. Old behavior still possible by specifying this as 'true' + (Erick Erickson) + +* SOLR-3012: Move System.getProperty("type") in postData() to main() and add type argument so that + the client applications of SimplePostTool can set content type via method argument. (koji) + +* SOLR-2888: FSTSuggester refactoring: internal storage is now UTF-8, + external sorting (on disk) prevents OOMs even with large data sets + (the bottleneck is now FST construction), code cleanups and API cleanups. + (Dawid Weiss, Robert Muir) + +Bug Fixes +---------------------- +* SOLR-3187 SystemInfoHandler leaks filehandles (siren) + +* LUCENE-3820: Fixed invalid position indexes by reimplementing PatternReplaceCharFilter. + This change also drops real support for boundary characters -- all input is prebuffered + for pattern matching. (Dawid Weiss) + +* SOLR-3068: Fixed NPE in ThreadDumpHandler (siren) + +* SOLR-2912: Fixed File descriptor leak in ShowFileRequestHandler (Michael Ryan, shalin) + +* SOLR-2819: Improved speed of parsing hex entities in HTMLStripCharFilter + (Bernhard Berger, hossman) + +* SOLR-2509: StringIndexOutOfBoundsException in the spellchecker collate when the term contains + a hyphen. (Thomas Gambier caught the bug, Steffen Godskesen did the patch, via Erick Erickson) + +* SOLR-2955: Fixed IllegalStateException when querying with group.sort=score desc in sharded + environment. (Steffen Elberg Godskesen, Martijn van Groningen) + +* SOLR-2956: Fixed inconsistencies in the flags (and flag key) reported by + the LukeRequestHandler (hossman) + +* SOLR-1730: Made it clearer when a core failed to load as well as better logging when the + QueryElevationComponent fails to properly initialize (gsingers) + +* SOLR-1520: QueryElevationComponent now supports non-string ids (gsingers) + +* SOLR-3024: Fixed JSONTestUtil.matchObj, in previous releases it was not + respecting the 'delta' arg (David Smiley via hossman) + +* SOLR-2542: Fixed DIH Context variables which were broken for all scopes other + then SCOPE_ENTITY (Linbin Chen & Frank Wesemann via hossman) + +* SOLR-3042: Fixed Maven Jetty plugin configuration. + (David Smiley via Steve Rowe) + +* SOLR-2970: CSV ResponseWriter returns fields defined as stored=false in schema (janhoy) + +* LUCENE-3690, LUCENE-2208, SOLR-882, SOLR-42: Re-implemented + HTMLStripCharFilter as a JFlex-generated scanner and moved it to + lucene/contrib/analyzers/common/. See below for a list of bug fixes and + other changes. To get the same behavior as HTMLStripCharFilter in Solr + version 3.5 and earlier (including the bugs), use LegacyHTMLStripCharFilter, + which is the previous implementation. + + Behavior changes from the previous version: + + - Known offset bugs are fixed. + - The "Mark invalid" exceptions reported in SOLR-1283 are no longer + triggered (the bug is still present in LegacyHTMLStripCharFilter). + - The character entity "'" is now always properly decoded. + - More cases of <script> tags are now properly stripped. + - CDATA sections are now handled properly. + - Valid tag name characters now include the supplementary Unicode characters + from Unicode character classes [:ID_Start:] and [:ID_Continue:]. + - Uppercase character entities """, "©", ">", "<", "®", + and "&" are now recognized and handled as if they were in lowercase. + - The REPLACEMENT CHARACTER U+FFFD is now used to replace numeric character + entities for unpaired UTF-16 low and high surrogates (in the range + [U+D800-U+DFFF]). + - Properly paired numeric character entities for UTF-16 surrogates are now + converted to the corresponding code units. + - Opening tags with unbalanced quotation marks are now properly stripped. + - Literal "<" and ">" characters in opening tags, regardless of whether they + appear inside quotation marks, now inhibit recognition (and stripping) of + the tags. The only exception to this is for values of event-handler + attributes, e.g. "onClick", "onLoad", "onSelect". + - A newline '\n' is substituted instead of a space for stripped HTML markup. + - Nothing is substituted for opening and closing inline tags - they are + simply removed. The list of inline tags is (case insensitively): <a>, + <abbr>, <acronym>, <b>, <basefont>, <bdo>, <big>, <cite>, <code>, <dfn>, + <em>, <font>, <i>, <img>, <input>, <kbd>, <label>, <q>, <s>, <samp>, + <select>, <small>, <span>, <strike>, <strong>, <sub>, <sup>, <textarea>, + <tt>, <u>, and <var>. + - HTMLStripCharFilterFactory now handles HTMLStripCharFilter's "escapedTags" + feature: opening and closing tags with the given names, including any + attributes and their values, are left intact in the output. + (Steve Rowe) + +* LUCENE-3717: Fixed offset bugs in TrimFilter, WordDelimiterFilter, and + HyphenatedWordsFilter where they would create invalid offsets in + some situations, leading to problems in highlighting. (Robert Muir) + +* SOLR-2280: commitWithin ignored for a delete query (Juan Grande via janhoy) + +* SOLR-3073: Fixed 'Invalid UUID string' error when having an UUID field as + the unique key and executing a distributed grouping request. (Devon Krisman, Martijn van Groningen) + +* SOLR-3084: Fixed initialization error when using + <queryResponseWriter default="true" ... /> (Bernd Fehling and hossman) + +* SOLR-3109: Fixed numerous redundant shard requests when using distributed grouping. + (rblack via Martijn van Groningen) + +* SOLR-3052: Fixed typo in distributed grouping parameters. + (Martijn van Groningen, Grant Ingersoll) + +* SOLR-2909: Add support for ResourceLoaderAware tokenizerFactories in synonym + filter factories. (Tom Klonikowski, Jun Ohtani via Koji Sekiguchi) + +* SOLR-3168: ReplicationHandler "numberToKeep" & "maxNumberOfBackups" parameters + would keep only 1 backup, even if more than 1 was specified (Neil Hooey, James Dyer) + +* SOLR-3009: hitGrouped.vm isn't shipped with 3.x (ehatcher, janhoy) + +* SOLR-3195: timeAllowed is ignored for grouping queries + (Russell Black via Martijn van Groningen) + +* SOLR-2124: Do not log stack traces for "Service Disabled" / 503 Exceptions (PingRequestHandler, etc) + (James Dyer, others) + +* SOLR-3260: DataImportHandler: ScriptTransformer gives better error messages when + problems arise on initialization (no Script Engine, invalid script, etc). (James Dyer) + +* SOLR-2959: edismax now respects the magic fields '_val_' and '_query_' + (Michael Watts, hossman) + +* SOLR-3074: fix SolrPluginUtils.docListToSolrDocumentList to respect the + list of fields specified. This fix also deprecates + DocumentBuilder.loadStoredFields which is not used anywhere in Solr, + and was fundamentally broken/bizarre. + (hossman, Ahmet Arslan) + +* SOLR-2291: fix JSONWriter to respect field list when writing SolrDocuments + (Ahmet Arslan via hossman) + +* SOLR-3264: Fix CoreContainer and SolrResourceLoader logging to be more + clear about when SolrCores are being created, and stop misleading people + about SolrCore instanceDir's being the "Solr Home Dir" (hossman) + +* SOLR-3046: Fix whitespace typo in DIH response "Time taken" (hossman) + +* SOLR-3261: Fix edismax to respect query operators when literal colons + are used in query string. (Juan Grande via hossman) + +* SOLR-3226: Fix SignatureUpdateProcessor to no longer ignore non-String + field values (Spyros Kapnissis, hossman) + +* SOLR-3200: Fix SignatureUpdateProcessor "all fields" mode to use all + fields of each document instead of the fields specified by the first + document indexed (Spyros Kapnissis via hossman) + +* SOLR-3316: Distributed grouping failed when rows parameter was set to 0 and + sometimes returned a wrong hit count as matches. (Cody Young, Martijn van Groningen) + +* SOLR-3107: contrib/langid: When using the LangDetect implementation of + langid, set the random seed to 0, so that the same document is detected as + the same language with the same probability every time. + (Christian Moen via rmuir) + +* SOLR-2937: Configuring the number of contextual snippets used for + search results clustering. The hl.snippets parameter is now respected + by the clustering plugin, can be overridden by carrot.summarySnippets + if needed (Stanislaw Osinski). + +* SOLR-2938: Clustering on multiple fields. The carrot.title and + carrot.snippet can now take comma- or space-separated lists of + field names to cluster (Stanislaw Osinski). + +* SOLR-2939: Clustering of multilingual search results. The document's + language field be passed in the carrot.lang parameter, the carrot.lcmap + parameter enables mapping of language codes to ISO 639 (Stanislaw Osinski). + +* SOLR-2940: Passing values for custom Carrot2 fields to Clustering component. + The custom field mapping are defined using the carrot.custom parameter + (Stanislaw Osinski). + +* SOLR-2941: NullPointerException on clustering component initialization + when schema does not have a unique key field (Stanislaw Osinski). + +* SOLR-2942: ClassCastException when passing non-textual fields to + clustering component (Stanislaw Osinski). + + +Other Changes +---------------------- +* SOLR-2922: Upgrade commons-io and commons-lang to 2.1 and 2.6, respectively. (koji) + +* SOLR-2920: Refactor frequent conditional use of DefaultSolrParams and + AppendedSolrParams into factory methods. + (David Smiley via hossman) + +* SOLR-3032: Deprecate logOnce from SolrException logOnce and all the supporting + structure will disappear in 4.0. Errors should be caught and logged at the + top-most level or logged and NOT propagated up the chain. (Erick Erickson) + +* SOLR-2718: Add ability to lazy load response writers, defined with startup="lazy". + (ehatcher) + +* SOLR-2901: Upgrade Solr to Tika 1.0 (janhoy) + +* SOLR-3059: Example XSL stylesheet for indexing query result XML (janhoy) + +* SOLR-3097, SOLR-3105: Add analysis configurations for different languages to + the example. (Christian Moen, Robert Muir) + +* SOLR-3005: Default QueryResponseWriters are now initialized via init() with an empty + NamedList. (Gasol Wu, Chris Male) + +* SOLR-3140: Upgrade schema version to 1.5, where omitNorms defaults to "true" for all + primitive (non-analyzed) field types such as int, float, date, bool, string.. (janhoy) + +* SOLR-3077: Better error messages when attempting to use "blank" field names + (Antony Stubbs via hossman) + +* SOLR-2712: expecting fl=score to return all fields is now deprecated. + In solr 4.0, this will only return the score. (ryan) + +* SOLR-3156: Check for Lucene directory locks at startup. In previous versions + this check was only performed during modifying (e.g. adding and deleting + documents) the index. (Luca Cavanna via Martijn van Groningen) + +* SOLR-1052: Deprecated <indexDefaults> and <mainIndex> in solrconfig.xml + From now, all settings go in the new <indexConfig> tag, and some defaults are + changed: useCompoundFile=false, ramBufferSizeMB=32, lockType=native, so that + the effect of NOT specifying <indexConfig> at all gives same result as the + example config used to give in 3.5 (janhoy, gsingers) + +* SOLR-3294: In contrib/clustering/lib/, replaced the manually retrowoven + Java 1.5-compatible carrot2-core-3.5.0.jar (which is not publicly available, + except from the Solr Subversion repository), with newly released Java + 1.5-compatible carrot2-core-3.5.0.1.jar (hosted on the Maven Central + repository). Also updated dependencies jackson-core-asl and + jackson-mapper-asl (both v1.5.2 -> v1.7.4). (Dawid Weiss, Steve Rowe) + +* SOLR-3295: netcdf jar is excluded from the binary release (and disabled in + ivy.xml) because it requires java 6. If you want to parse this content with + extracting request handler and are willing to use java 6, just add the jar. + (rmuir) + +* SOLR-3142: DIH Imports no longer default optimize to true, instead false. + If you want to force all segments to be merged into one, you can specify + this parameter yourself. NOTE: this can be very expensive operation and + usually does not make sense for delta-imports. (Robert Muir) + +Build +---------------------- +* SOLR-2487: Add build target to package war without slf4j jars (janhoy) + +* SOLR-3112: Fix tests not to write to src/test-files (Luca Cavanna via Robert Muir) + +* LUCENE-3753: Restructure the Solr build system. (Steve Rowe) + +* SOLR-3204: The packaged pre-release artifact of Commons CSV used the original + package name (org.apache.commons.csv). This created a compatibility issue as + the Apache Commons team works toward an official release of Commons CSV. + The source of Commons CSV was added under a separate package name to the + Solr source code. (Uwe Schindler, Chris Male, Emmanuel Bourg) + +* LUCENE-3930: Changed build system to use Apache Ivy for retrival of 3rd + party JAR files. Please review README.txt for instructions. + (Robert Muir, Chris Male, Uwe Schindler, Steven Rowe, Hossman) + +================== 3.5.0 ================== + +New Features +---------------------- +* SOLR-2749: Add boundary scanners for FastVectorHighlighter. <boundaryScanner/> + can be specified with a name in solrconfig.xml, and use hl.boundaryScanner=name + parameter to specify the named <boundaryScanner/>. (koji) + +* SOLR-2066,SOLR-2776: Added support for distributed grouping. + (Martijn van Groningen, Jasper van Veghel, Matt Beaumont) + +* SOLR-2769: Added factory for the new Hunspell stemmer capable of doing stemming + for 99 languages (janhoy, cmale) + +* SOLR-1979: New contrib "langid". Adds language identification capabilities as an + Update Processor, using Tika's LanguageIdentifier or Cybozu language-detection + library (janhoy, Tommaso Teofili, gsingers) + +* SOLR-2818: Added before/after count response parsing support for range facets in + SolrJ. (Bernhard Frauendienst via Martijn van Groningen) + +* SOLR-2276: Add support for cologne phonetic to PhoneticFilterFactory. + (Marc Pompl via rmuir) + +* SOLR-1926: Add hl.q parameter. (koji) + +* SOLR-2881: Numeric types now support sortMissingFirst/Last. This includes Trie and date types + (Ryan McKinley, Mike McCandless, Uwe Schindler, Erick Erickson) + +* SOLR-1023: StatsComponent now supports date fields and string fields. + (Chris Male, Mark Holland, Gunnlaugur Thor Briem, Ryan McKinley) + +* SOLR-2578: ReplicationHandler's backup command now supports a 'numberToKeep' + request param that can be used to delete all but the most recent N backups. + (James Dyer via hossman) + +* SOLR-2839: Add alternative implementation to contrib/langid supporting 53 + languages, based on http://code.google.com/p/language-detection/ (rmuir) + +Optimizations +---------------------- + +* SOLR-2742: SolrJ: Provide commitWithinMs as optional parameter for all add() methods, + making the feature more conveniently accessible for developers (janhoy) + +Bug Fixes +---------------------- +* SOLR-2748: The CommitTracker used for commitWith or autoCommit by maxTime + could commit too frequently and could block adds until a new searcher was + registered. (yonik) + +* SOLR-2726: Fixed NullPointerException when using spellcheck.q with Suggester. + (Bernd Fehling, valentin via rmuir) + +* SOLR-2772: Fixed Date parsing/formatting of years 0001-1000 (hossman) + +* SOLR-2763: Extracting update request handler throws exception and returns 400 + when zero-length file posted using multipart form post (janhoy) + +* SOLR-2780: Fixed issue where multi select facets didn't respect group.truncate parameter. + (Martijn van Groningen, Ramzi Alqrainy) + +* SOLR-2793: In rare cases (most likely during shutdown), a SolrIndexSearcher can be left + open if the executor rejects a task. (Mark Miller) + +* SOLR-2791: Replication: abortfetch command is broken if replication was started + by fetchindex command instead of a regular poll (Yury Kats via shalin) + +* SOLR-2861: Fix extremely rare race condition on commit that can result + in a NPE (yonik) + +* SOLR-2813: Fix HTTP error codes returned when requests contain strings that + can not be parsed as numbers for Trie fields. (Jeff Crump and hossman) + +* SOLR-2902: List of collations are wrong parsed in SpellCheckResponse causing + a wrong number of collation results in the response. + (Bastiaan Verhoef, James Dyer via Simon Willnauer) + +* SOLR-2875: Fix the incorrect url in DIH example tika-data-config.xml + (Shinichiro Abe via koji) + +Other Changes +---------------------- + +* SOLR-2750: Make both "update.chain" and the deprecated "update.param" work + consistently everywhere; see also SOLR-2105. (Mark Miller, janhoy) + +* LUCENE-3410: Deprecated the WordDelimiterFilter constructors accepting multiple + ints masquerading as booleans. Preferred constructor now accepts a single int + bitfield (Chris Male) + +* SOLR-2758: Moved ConcurrentLRUCache from o.a.s.common.util package in the solrj + module to the o.a.s.util package in the Solr core module. + (David Smiley via Steve Rowe) + +* SOLR-2766: Package individual javadoc sites for solrj and test-framework. + (Steve Rowe, Mike McCandless) + +* SOLR-2771: Solr modules' tests should not depend on solr-core test classes; + move BufferingRequestProcessor from solr-core tests to test-framework so that + the Solr Cell module can use it. (janhoy, Steve Rowe) + +* LUCENE-3457: Upgrade commons-compress to 1.2 (Doron Cohen) + +* SOLR-2757: min() and max() functions now support an arbitrary number of + ValueSources (Bill Bell via hossman) + +* SOLR-2372: Upgrade Solr to Tika 0.10 (janhoy) + +* SOLR-2792: Allow case insensitive Hunspell stemming (janhoy, rmuir) + +* SOLR-2862: More explicit lexical resources location logged if Carrot2 clustering + extension is used. Fixed solr. impl. of IResource and IResourceLookup. (Dawid Weiss) + +* SOLR-2849: Fix dependencies in Maven POMs. (David Smiley via Steve Rowe) + +* SOLR-2591: Remove commitLockTimeout option from solrconfig.xml (Luca Cavanna via Martijn van Groningen) + +* SOLR-2746: Upgraded UIMA dependencies from *-2.3.1-SNAPSHOT.jar to *-2.3.1.jar. + + +================== 3.4.0 ================== + +Upgrading from Solr 3.3 +---------------------- + +* The Lucene index format has changed and as a result, once you upgrade, + previous versions of Solr will no longer be able to read your indices. + In a master/slave configuration, all searchers/slaves should be upgraded + before the master. If the master were to be updated first, the older + searchers would not be able to read the new index format. + +* Previous versions of Solr silently allow and ignore some contradictory + properties specified in schema.xml. For example: + - indexed="false" omitNorms="false" + - indexed="false" omitTermFreqAndPositions="false" + Field property validation has now been fixed, to ensure that + contradictions like these now generate error messages. If users + have existing schemas that generate one of these new "conflicting + 'false' field options for non-indexed field" error messages the + conflicting "omit*" properties can safely be removed, or changed to + "true" for consistent behavior with previous Solr versions. This + situation has now been fixed to cause an error on startup when these + contradictory options. See SOLR-2669. + +* FacetComponent no longer catches and embeds exceptions occurred during facet + processing, it throws HTTP 400 or 500 exceptions instead. + +New Features +---------------------- + +* SOLR-2540: CommitWithin as an Update Request parameter + You can now specify &commitWithin=N (ms) on the update request (janhoy) + +* SOLR-2458: post.jar enhanced to handle JSON, CSV and <optimize> (janhoy) + +* LUCENE-3234: add a new parameter hl.phraseLimit for FastVectorHighlighter speed up. + (Mike Sokolov via koji) + +* SOLR-2429: Ability to add cache=false to queries and query filters to avoid + using the filterCache or queryCache. A cost may also be specified and is used + to order the evaluation of non-cached filters from least to greatest cost . + For very expensive query filters (cost >= 100) if the query implements + the PostFilter interface, it will be used to obtain a Collector that is + checked only for documents that match the main query and all other filters. + The "frange" query now implements the PostFilter interface. (yonik) + +* SOLR-2630: Added new XsltUpdateRequestHandler that works like + XmlUpdateRequestHandler but allows to transform the POSTed XML document + using XSLT. This allows to POST arbitrary XML documents to the update + handler, as long as you also provide a XSL to transform them to a valid + Solr input document. (Upayavira, Uwe Schindler) + +* SOLR-2615: Log individual updates (adds and deletes) at the FINE level + before adding to the index. Fix a null pointer exception in logging + when there was no unique key. (David Smiley via yonik) + +* LUCENE-2048: Added omitPositions to the schema, so you can omit position + information while still indexing term frequencies. (rmuir) + +* SOLR-2584: add UniqFieldsUpdateProcessor that removes duplicate values in the + specified fields. (Elmer Garduno, koji) + +* SOLR-2670: Added NIOFSDirectoryFactory (yonik) + +* SOLR-2523: Added support in SolrJ to easily interact with range facets. + The range facet response can be parsed and is retrievable from the + QueryResponse class. The SolrQuery class has convenient methods for using + range facets. (Martijn van Groningen) + +* SOLR-2637: Added support for group result parsing in SolrJ. + (Tao Cheng, Martijn van Groningen) + +* SOLR-2665: Added post group faceting. Facet counts are based on the most + relevant document of each group matching the query. This feature has the + same impact on the StatsComponent. (Martijn van Groningen) + +* SOLR-2675: CoreAdminHandler now allows arbitrary properties to be + specified when CREATEing a new SolrCore using property.* request + params. (Yury Kats, hossman) + +* SOLR-2714: JSON update format - "null" field values are now dropped + instead of causing an exception. (Trygve Laugstøl, yonik) + + +Optimizations +---------------------- + +* LUCENE-3233: Improved memory usage, build time, and performance of + SynonymFilterFactory. (Mike McCandless, Robert Muir) + +Bug Fixes +---------------------- + +* SOLR-2625: TermVectorComponent throws NPE if TF-IDF option is used without DF + option. (Daniel Erenrich, Simon Willnauer) + +* SOLR-2631: PingRequestHandler should not allow to ping itself using "qt" + param to prevent infinite loop. (Edoardo Tosca, Uwe Schindler) + +* SOLR-2636: Fix explain functionality for negative queries. (Tom Hill via yonik) + +* SOLR-2538: Range Faceting on long/double fields could overflow if values + bigger then the max int/float were used. + (Erbi Hanka, hossman) + +* SOLR-2230: CommonsHttpSolrServer.addFile could not be used to send + multiple files in a single request. + (Stephan Günther, hossman) + +* SOLR-2541: PluginInfos was not correctly parsing <long/> tags when + initializing plugins + (Frank Wesemann, hossman) + +* SOLR-2623: Solr JMX MBeans do not survive core reloads (Alexey Serba, shalin) + +* Fixed grouping bug when start is bigger than rows and format is simple that zero documents are returned even + if there are documents to display. (Martijn van Groningen, Nikhil Chhaochharia) + +* SOLR-2564: Fixed ArrayIndexOutOfBoundsException when using simple format and + start > 0 (Martijn van Groningen, Matteo Melli) + +* SOLR-2642: Fixed sorting by function when using grouping. (Thomas Heigl, Martijn van Groningen) + +* SOLR-2535: REGRESSION: in Solr 3.x and trunk the admin/file handler + fails to show directory listings (David Smiley, Peter Wolanin via Erick Erickson) + +* SOLR-2545: ExternalFileField file parsing would fail if any key + contained an "=" character. It now only looks for the last "=" delimiter + prior to the float value. + (Markus Jelsma, hossman) + +* SOLR-2662: When Solr is configured to have no queryResultCache, the + "start" parameter was not honored and the documents returned were + 0 through start+offset. (Markus Jelsma, yonik) + +* SOLR-2669: Fix backwards validation of field properties in + SchemaField.calcProps (hossman) + +* SOLR-2676: Add "welcome-file-list" to solr.war so admin UI works correctly + in servlet containers such as WebSphere that do not use a default list + (Jay R. Jaeger, hossman) + +* SOLR-2606: Fixed sort parsing of fields containing punctuation that + failed due to sort by function changes introduced in SOLR-1297 + (Mitsu Hadeishi, hossman) + +* SOLR-2706: contrib/clustering: The carrot.lexicalResourcesDir parameter + now works with absolute directories (Stanislaw Osinski) + +* SOLR-2692: contrib/clustering: Typo in param name fixed: "carrot.fragzise" + changed to "carrot.fragSize" (Stanislaw Osinski). + +* SOLR-2644: When using DIH with threads=2 the default logging is set too high + (Bill Bell via shalin) + +* SOLR-2492: DIH does not commit if only deletes are processed + (James Dyer via shalin) + +* SOLR-2186: DataImportHandler's multi-threaded option throws NPE + (Lance Norskog, Frank Wesemann, shalin) + +* SOLR-2655: DIH multi threaded mode does not resolve attributes correctly + (Frank Wesemann, shalin) + +* SOLR-2695: DIH: Documents are collected in unsynchronized list in + multi-threaded debug mode (Michael McCandless, shalin) + +* SOLR-2668: DIH multithreaded mode does not rollback on errors from + EntityProcessor (Frank Wesemann, shalin) + +Other Changes +---------------------- + +* SOLR-2629: Eliminate deprecation warnings in some JSPs. + (Bernd Fehling, hossman) + +* SOLR-2743: Remove commons logging from contrib/extraction. (koji) + + +Build +---------------------- + +* SOLR-2452,SOLR-2653,LUCENE-3323,SOLR-2659,LUCENE-3329,SOLR-2666: + Rewrote the Solr build system: + - Integrated more fully with the Lucene build system: generalized the + Lucene build system and eliminated duplication. + - Converted all Solr contribs to the Lucene/Solr conventional src/ layout: + java/, resources/, test/, and test-files/<contrib-name>. + - Created a new Solr-internal module named "core" by moving the java/, + test/, and test-files/ directories from solr/src/ to solr/core/src/. + - Merged solr/src/webapp/src/ into solr/core/src/java/. + - Eliminated solr/src/ by moving all its directories up one level; + renamed solr/src/site/ to solr/site-src/ because solr/site/ already + exists. + - Merged solr/src/common/ into solr/solrj/src/java/. + - Moved o.a.s.client.solrj.* and o.a.s.common.* tests from + solr/src/test/ to solr/solrj/src/test/. + - Made the solrj tests not depend on the solr core tests by moving + some classes from solr/src/test/ to solr/test-framework/src/java/. + - Each internal module (core/, solrj/, test-framework/, and webapp/) + now has its own build.xml, from which it is possible to run + module-specific targets. solr/build.xml delegates all build + tasks (via <ant dir="internal-module-dir"> calls) to these + modules' build.xml files. + (Steve Rowe, Robert Muir) + +* LUCENE-3406: Add ant target 'package-local-src-tgz' to Lucene and Solr + to package sources from the local working copy. + (Seung-Yeoul Yang via Steve Rowe) + +Documentation +---------------------- + +================== 3.3.0 ================== + +Upgrading from Solr 3.2.0 +---------------------- + * SolrCore's CloseHook API has been changed in a backward-incompatible way. It + has been changed from an interface to an abstract class. Any custom + components which use the SolrCore.addCloseHook method will need to + be modified accordingly. To migrate, put your old CloseHook#close impl into + CloseHook#preClose. + +New Features +---------------------- + +* SOLR-2378: A new, automaton-based, implementation of suggest (autocomplete) + component, offering an order of magnitude smaller memory consumption + compared to ternary trees and jaspell and very fast lookups at runtime. + (Dawid Weiss) + +* SOLR-2400: Field- and DocumentAnalysisRequestHandler now provide a position + history for each token, so you can follow the token through all analysis stages. + The output contains a separate int[] attribute containing all positions from + previous Tokenizers/TokenFilters (called "positionHistory"). + (Uwe Schindler) + +* SOLR-2524: (SOLR-236, SOLR-237, SOLR-1773, SOLR-1311) Grouping / Field collapsing + using the Lucene grouping contrib. The search result can be grouped by field and query. + (Martijn van Groningen, Emmanuel Keller, Shalin Shekhar Mangar, Koji Sekiguchi, + Iván de Prado, Ryan McKinley, Marc Sturlese, Peter Karich, Bojan Smid, + Charles Hornberger, Dieter Grad, Dmitry Lihachev, Doug Steigerwald, + Karsten Sperling, Michael Gundlach, Oleg Gnatovskiy, Thomas Traeger, + Harish Agarwal, yonik, Michael McCandless, Bill Bell) + +* SOLR-1331 -- Added a srcCore parameter to CoreAdminHandler's mergeindexes action + to merge one or more cores' indexes to a target core (shalin) + +* SOLR-2610 -- Add an option to delete index through CoreAdmin UNLOAD action (shalin) + +* SOLR-2480: Add ignoreTikaException flag to the extraction request handler so + that users can ignore TikaException but index meta data. + (Shinichiro Abe, koji) + +* SOLR-2582: Use uniqueKey for error log in UIMAUpdateRequestProcessor. + (Tommaso Teofili via koji) + +Optimizations +---------------------- + +* SOLR-2567: Solr now defaults to TieredMergePolicy. See http://s.apache.org/merging + for more information. (rmuir) + +Bug Fixes +---------------------- + +* SOLR-2519: Improve text_* fieldTypes in example schema.xml: improve + cross-language defaults for text_general; break out separate + English-specific fieldTypes (Jan Høydahl, hossman, Robert Muir, + yonik, Mike McCandless) + +* SOLR-2462: Fix extremely high memory usage problems with spellcheck.collate. + Separately, an additional spellcheck.maxCollationEvaluations (default=10000) + parameter is added to avoid excessive CPU time in extreme cases (e.g. long + queries with many misspelled words). (James Dyer via rmuir) + +* SOLR-2579: UIMAUpdateRequestProcessor ignore error fails if text.length() < 100. + (Elmer Garduno via koji) + +* SOLR-2581: UIMAToSolrMapper wrongly instantiates Type with reflection. + (Tommaso Teofili via koji) + +* SOLR-2551: Check dataimport.properties for write access (if delta-import is + supported in DIH configuration) before starting an import (C S, shalin) + +Other Changes +---------------------- + +* SOLR-2571: Add a commented out example of the spellchecker's thresholdTokenFrequency + parameter to the example solrconfig.xml, and also add a unit test for this feature. + (James Dyer via rmuir) + +* SOLR-2576: Deprecate SpellingResult.add(Token token, int docFreq), please use + SpellingResult.addFrequency(Token token, int docFreq) instead. + (James Dyer via rmuir) + +* SOLR-2574: Upgrade slf4j to v1.6.1 (shalin) + +* LUCENE-3204: The maven-ant-tasks jar is now included in the source tree; + users of the generate-maven-artifacts target no longer have to manually + place this jar in the Ant classpath. NOTE: when Ant looks for the + maven-ant-tasks jar, it looks first in its pre-existing classpath, so + any copies it finds will be used instead of the copy included in the + Lucene/Solr source tree. For this reason, it is recommeded to remove + any copies of the maven-ant-tasks jar in the Ant classpath, e.g. under + ~/.ant/lib/ or under the Ant installation's lib/ directory. (Steve Rowe) + +* SOLR-2611: Fix typos in the example configuration (Eric Pugh via rmuir) + +================== 3.2.0 ================== +Versions of Major Components +--------------------- +Apache Lucene trunk +Apache Tika 0.8 +Carrot2 3.4.2 + + +Upgrading from Solr 3.1 +---------------------- + +* The updateRequestProcessorChain for a RequestHandler is now defined + with update.chain rather than update.processor. The latter still works, + but has been deprecated. + +* <uimaConfig/> just beneath <config> ... </config> is no longer supported. + It should move to UIMAUpdateRequestProcessorFactory setting. + See contrib/uima/README.txt for more details. (SOLR-2436) + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-2496: Add ability to specify overwrite and commitWithin as request + parameters (e.g. specified in the URL) when using the JSON update format, + and added a simplified format for specifying multiple documents. + Example: [{"id":"doc1"},{"id":"doc2"}] + (yonik) + +* SOLR-2113: Add TermQParserPlugin, registered as "term". This is useful + when generating filter queries from terms returned from field faceting or + the terms component. Example: fq={!term f=weight}1.5 (hossman, yonik) + +* SOLR-1915: DebugComponent now supports using a NamedList to model + Explanation objects in its responses instead of + Explanation.toString (hossman) + +* SOLR-2448: Search results clustering updates: bisecting k-means + clustering algorithm added, loading of Carrot2 stop words from + <solr.home>/conf/carrot2 (SOLR-2449), using Solr's stopwords.txt + for clustering (SOLR-2450), output of cluster scores (SOLR-2505) + (Stanislaw Osinski, Dawid Weiss). + +* SOLR-2503: extend UIMAUpdateRequestProcessorFactory mapping function to + map feature value to dynamicField. (koji) + +* SOLR-2512: add ignoreErrors flag to UIMAUpdateRequestProcessorFactory so + that users can ignore exceptions in AE. (Tommaso Teofili, koji) + +Optimizations +---------------------- + +Bug Fixes +---------------------- + +* SOLR-2445: Change the default qt to blank in form.jsp, because there is no "standard" + request handler unless you have it in your solrconfig.xml explicitly. (koji) + +* SOLR-2455: Prevent double submit of forms in admin interface. + (Jeffrey Chang via uschindler) + +* SOLR-2464: Fix potential slowness in QueryValueSource (the query() function) when + the query is very sparse and may not match any documents in a segment. (yonik) + +* SOLR-2469: When using java replication with replicateAfter=startup, the first + commit point on server startup is never removed. (yonik) + +* SOLR-2466: SolrJ's CommonsHttpSolrServer would retry requests on failure, regardless + of the configured maxRetries, due to HttpClient having its own retry mechanism + by default. The retryCount of HttpClient is now set to 0, and SolrJ does + the retry. (yonik) + +* SOLR-2409: edismax parser - treat the text of a fielded query as a literal if the + fieldname does not exist. For example Mission: Impossible should not search on + the "Mission" field unless it's a valid field in the schema. (Ryan McKinley, yonik) + +* SOLR-2403: facet.sort=index reported incorrect results for distributed search + in a number of scenarios when facet.mincount>0. This patch also adds some + performance/algorithmic improvements when (facet.sort=count && facet.mincount=1 + && facet.limit=-1) and when (facet.sort=index && facet.mincount>0) (yonik) + +* SOLR-2333: The "rename" core admin action does not persist the new name to solr.xml + (Rasmus Hahn, Paul R. Brown via Mark Miller) + +* SOLR-2390: Performance of usePhraseHighlighter is terrible on very large Documents, + regardless of hl.maxDocCharsToAnalyze. (Mark Miller) + +* SOLR-2474: The helper TokenStreams in analysis.jsp and AnalysisRequestHandlerBase + did not clear all attributes so they displayed incorrect attribute values for tokens + in later filter stages. (uschindler, rmuir, yonik) + +* SOLR-2467: Fix <analyzer class="..." /> initialization so any errors + are logged properly. (hossman) + +* SOLR-2493: SolrQueryParser was fixed to not parse the SolrConfig DOM tree on each + instantiation which is a huge slowdown. (Stephane Bailliez via uschindler) + +* SOLR-2495: The JSON parser could hang on corrupted input and could fail + to detect numbers that were too large to fit in a long. (yonik) + +* SOLR-2520: Make JSON response format escape \u2029 as well as \u2028 + in strings since those characters are not valid in javascript strings + (although they are valid in JSON strings). (yonik) + +* SOLR-2536: Add ReloadCacheRequestHandler to fix ExternalFileField bug (if reopenReaders + set to true and no index segments have been changed, commit cannot trigger reload + external file). (koji) + +* SOLR-2539: VectorValueSource.floatVal incorrectly used byteVal on sub-sources. + (Tom Liu via yonik) + +* SOLR-2554: RandomSortField didn't work when used in a function query. (yonik) + + +Other Changes +---------------------- + +* SOLR-2061: Pull base tests out into a new Solr Test Framework module, + and publish binary, javadoc, and source test-framework jars. + (Drew Farris, Robert Muir, Steve Rowe) + +* SOLR-2105: Rename RequestHandler param 'update.processor' to 'update.chain'. + (Jan Høydahl via Mark Miller) + +* SOLR-2485: Deprecate BaseResponseWriter, GenericBinaryResponseWriter, and + GenericTextResponseWriter. These classes will be removed in 4.0. (ryan) + +* SOLR-2451: Enhance assertJQ to allow individual tests to specify the + tolerance delta used in numeric equalities. This allows for slight + variance in asserting score comparisons in unit tests. + (David Smiley, Chris Hostetter) + +* SOLR-2528: Remove default="true" from HtmlEncoder in example solrconfig.xml, + because html encoding confuses non-ascii users. (koji) + +* SOLR-2387: add mock annotators for improved testing in contrib/uima, + (Tommaso Teofili via rmuir) + +* SOLR-2436: move uimaConfig to under the uima's update processor in + solrconfig.xml. (Tommaso Teofili, koji) + +Build +---------------------- + +* LUCENE-3006: Building javadocs will fail on warnings by default. Override with -Dfailonjavadocwarning=false (sarowe, gsingers) + + +Documentation +---------------------- + + +================== 3.1.0 ================== +Versions of Major Components +--------------------- +Apache Lucene 3.1.0 +Apache Tika 0.8 +Carrot2 3.4.2 +Velocity 1.6.1 and Velocity Tools 2.0-beta3 +Apache UIMA 2.3.1-SNAPSHOT + + +Upgrading from Solr 1.4 +---------------------- + +* The Lucene index format has changed and as a result, once you upgrade, + previous versions of Solr will no longer be able to read your indices. + In a master/slave configuration, all searchers/slaves should be upgraded + before the master. If the master were to be updated first, the older + searchers would not be able to read the new index format. + +* The Solr JavaBin format has changed as of Solr 3.1. If you are using the + JavaBin format, you will need to upgrade your SolrJ client. (SOLR-2034) + +* The experimental ALIAS command has been removed (SOLR-1637) + +* Using solr.xml is recommended for single cores also (SOLR-1621) + +* Old syntax of <highlighting> configuration in solrconfig.xml + is deprecated (SOLR-1696) + +* The deprecated HTMLStripReader, HTMLStripWhitespaceTokenizerFactory and + HTMLStripStandardTokenizerFactory were removed. To strip HTML tags, + HTMLStripCharFilter should be used instead, and it works with any + Tokenizer of your choice. (SOLR-1657) + +* Field compression is no longer supported. Fields that were formerly + compressed will be uncompressed as index segments are merged. For + shorter fields, this may actually be an improvement, as the compression + used was not very good for short text. Some indexes may get larger though. + +* SOLR-1845: The TermsComponent response format was changed so that the + "terms" container is a map instead of a named list. This affects + response formats like JSON, but not XML. (yonik) + +* SOLR-1876: All Analyzers and TokenStreams are now final to enforce + the decorator pattern. (rmuir, uschindler) + +* LUCENE-2608: Added the ability to specify the accuracy on a per request basis. + It is recommended that implementations of SolrSpellChecker should change over to the new SolrSpellChecker + methods using the new SpellingOptions class, but are not required to. While this change is + backward compatible, the trunk version of Solr has already dropped support for all but the SpellingOptions method. (gsingers) + +* readercycle script was removed. (SOLR-2046) + +* In previous releases, sorting or evaluating function queries on + fields that were "multiValued" (either by explicit declaration in + schema.xml or by implict behavior because the "version" attribute on + the schema was less then 1.2) did not generally work, but it would + sometimes silently act as if it succeeded and order the docs + arbitrarily. Solr will now fail on any attempt to sort, or apply a + function to, multi-valued fields + +* The DataImportHandler jars are no longer included in the solr + WAR and should be added in Solr's lib directory, or referenced + via the <lib> directive in solrconfig.xml. + + +Detailed Change List +---------------------- + +New Features +---------------------- + +* SOLR-1302: Added several new distance based functions, including + Great Circle (haversine), Manhattan, Euclidean and String (using the + StringDistance methods in the Lucene spellchecker). + Also added geohash(), deg() and rad() convenience functions. + See http://wiki.apache.org/solr/FunctionQuery. (gsingers) + +* SOLR-1553: New dismax parser implementation (accessible as "edismax") + that supports full lucene syntax, improved reserved char escaping, + fielded queries, improved proximity boosting, and improved stopword + handling. Note: status is experimental for now. (yonik) + +* SOLR-1574: Add many new functions from java Math (e.g. sin, cos) (yonik) + +* SOLR-1569: Allow functions to take in literal strings by modifying the + FunctionQParser and adding LiteralValueSource (gsingers) + +* SOLR-1571: Added unicode collation support though Lucene's CollationKeyFilter + (Robert Muir via shalin) + +* SOLR-785: Distributed Search support for SpellCheckComponent + (Matthew Woytowitz, shalin) + +* SOLR-1625: Add regexp support for TermsComponent (Uri Boness via noble) + +* SOLR-1297: Add sort by Function capability (gsingers, yonik) + +* SOLR-1139: Add TermsComponent Query and Response Support in SolrJ (Matt Weber via shalin) + +* SOLR-1177: Distributed Search support for TermsComponent (Matt Weber via shalin) + +* SOLR-1621, SOLR-1722: Allow current single core deployments to be specified by solr.xml (Mark Miller , noble) + +* SOLR-1532: Allow StreamingUpdateSolrServer to use a provided HttpClient (Gabriele Renzi via shalin) + +* SOLR-1653: Add PatternReplaceCharFilter (koji) + +* SOLR-1131: FieldTypes can now output multiple Fields per Type and still be searched. This can be handy for hiding the details of a particular + implementation such as in the spatial case. (Chris Mattmann, shalin, noble, gsingers, yonik) + +* SOLR-1586: Add support for Geohash and Spatial Tile FieldType (Chris Mattmann, gsingers) + +* SOLR-1697: PluginInfo should load plugins w/o class attribute also (noble) + +* SOLR-1268: Incorporate FastVectorHighlighter (koji) + +* SOLR-1750: SolrInfoMBeanHandler added for simpler programmatic access + to info currently available from registry.jsp and stats.jsp + (ehatcher, hossman) + +* SOLR-1815: SolrJ now preserves the order of facet queries. (yonik) + +* SOLR-1677: Add support for choosing the Lucene Version for Lucene components within + Solr. (Uwe Schindler, Mark Miller) + +* SOLR-1379: Add RAMDirectoryFactory for non-persistent in memory index storage. + (Alex Baranov via yonik) + +* SOLR-1857: Synced Solr analysis with Lucene 3.1. Added KeywordMarkerFilterFactory + and StemmerOverrideFilterFactory, which can be used to tune stemming algorithms. + Added factories for Bulgarian, Czech, Hindi, Turkish, and Wikipedia analysis. Improved the + performance of SnowballPorterFilterFactory. (rmuir) + +* SOLR-1657: Converted remaining TokenStreams to the Attributes-based API. All Solr + TokenFilters now support custom Attributes, and some have improved performance: + especially WordDelimiterFilter and CommonGramsFilter. (rmuir, cmale, uschindler) + +* SOLR-1740: ShingleFilterFactory supports the "minShingleSize" and "tokenSeparator" + parameters for controlling the minimum shingle size produced by the filter, and + the separator string that it uses, respectively. (Steven Rowe via rmuir) + +* SOLR-744: ShingleFilterFactory supports the "outputUnigramsIfNoShingles" + parameter, to output unigrams if the number of input tokens is fewer than + minShingleSize, and no shingles can be generated. + (Chris Harris via Steven Rowe) + +* SOLR-1923: PhoneticFilterFactory now has support for the + Caverphone algorithm. (rmuir) + +* SOLR-1957: The VelocityResponseWriter contrib moved to core. + Example search UI now available at http://localhost:8983/solr/browse + (ehatcher) + +* SOLR-1974: Add LimitTokenCountFilterFactory. (koji) + +* SOLR-1966: QueryElevationComponent can now return just the included results in the elevation file (gsingers, yonik) + +* SOLR-1556: TermVectorComponent now supports per field overrides. Also, it now throws an error + if passed in fields do not exist and warnings + if fields that do not have term vector options (termVectors, offsets, positions) + that align with the schema declaration. It also + will now return warnings about (gsingers) + +* SOLR-1985: FastVectorHighlighter: add wrapper class for Lucene's SingleFragListBuilder (koji) + +* SOLR-1984: Add HyphenationCompoundWordTokenFilterFactory. (PB via rmuir) + +* SOLR-397: Date Faceting now supports a "facet.date.include" param + for specifying when the upper & lower end points of computed date + ranges should be included in the range. Legal values are: "all", + "lower", "upper", "edge", and "outer". For backwards compatibility + the default value is the set: [lower,upper,edge], so that all ranges + between start and end are inclusive of their endpoints, but the + "before" and "after" ranges are not. + +* SOLR-945: JSON update handler that accepts add, delete, commit + commands in JSON format. (Ryan McKinley, yonik) + +* SOLR-2015: Add a boolean attribute autoGeneratePhraseQueries to TextField. + autoGeneratePhraseQueries="true" (the default) causes the query parser to + generate phrase queries if multiple tokens are generated from a single + non-quoted analysis string. For example WordDelimiterFilter splitting text:pdp-11 + will cause the parser to generate text:"pdp 11" rather than (text:PDP OR text:11). + Note that autoGeneratePhraseQueries="true" tends to not work well for non whitespace + delimited languages. (yonik) + +* SOLR-1925: Add CSVResponseWriter (use wt=csv) that returns the list of documents + in CSV format. (Chris Mattmann, yonik) + +* SOLR-1240: "Range Faceting" has been added. This is a generalization + of the existing "Date Faceting" logic so that it now supports any + all stock numeric field types that support range queries in addition + to dates. facet.date is now deprecated in favor of this generalized mechanism. + (Gijs Kunze, hossman) + +* SOLR-2021: Add SolrEncoder plugin to Highlighter. (koji) + +* SOLR-2030: Make FastVectorHighlighter use of SolrEncoder. (koji) + +* SOLR-2053: Add support for custom comparators in Solr spellchecker, per LUCENE-2479 (gsingers) + +* SOLR-2049: Add hl.multiValuedSeparatorChar for FastVectorHighlighter, per LUCENE-2603. (koji) + +* SOLR-2059: Add "types" attribute to WordDelimiterFilterFactory, which + allows you to customize how WordDelimiterFilter tokenizes text with + a configuration file. (Peter Karich, rmuir) + +* SOLR-2099: Add ability to throttle rsync based replication using rsync option --bwlimit. + (Brandon Evans via koji) + +* SOLR-1316: Create autosuggest component. + (Ankul Garg, Jason Rutherglen, Shalin Shekhar Mangar, Grant Ingersoll, Robert Muir, ab) + +* SOLR-1568: Added "native" filtering support for PointType, GeohashField. Added LatLonType with filtering support too. See + http://wiki.apache.org/solr/SpatialSearch and the example. Refactored some items in Lucene spatial. + Removed SpatialTileField as the underlying CartesianTier is broken beyond repair and is going to be moved. (gsingers) + +* SOLR-2128: Full parameter substitution for function queries. + Example: q=add($v1,$v2)&v1=mul(popularity,5)&v2=20.0 + (yonik) + +* SOLR-2133: Function query parser can now parse multiple comma separated + value sources. It also now fails if there is extra unexpected text + after parsing the functions, instead of silently ignoring it. + This allows expressions like q=dist(2,vector(1,2),$pt)&pt=3,4 (yonik) + +* SOLR-2157: Suggester should return alpha-sorted results when onlyMorePopular=false (ab) + +* SOLR-2010: Added ability to verify that spell checking collations have + actual results in the index. (James Dyer via gsingers) + +* SOLR-2188: Added "maxTokenLength" argument to the factories for ClassicTokenizer, + StandardTokenizer, and UAX29URLEmailTokenizer. (Steven Rowe) + +* SOLR-2129: Added a Solr module for dynamic metadata extraction/indexing with Apache UIMA. + See contrib/uima/README.txt for more information. (Tommaso Teofili via rmuir) + +* SOLR-2325: Allow tagging and exclusion of main query for faceting. (yonik) + +* SOLR-2263: Add ability for RawResponseWriter to stream binary files as well as + text files. (Eric Pugh via yonik) + +* SOLR-860: Add debug output for MoreLikeThis. (koji) + +* SOLR-1057: Add PathHierarchyTokenizerFactory. (ryan, koji) + +* SOLR-1804: Re-enabled clustering component on trunk, updated to latest + version of Carrot2. No more LGPL run-time dependencies. This release of + C2 also does not have a specific Lucene dependency. + (Stanislaw Osinski, gsingers) + +* SOLR-2282: Add distributed search support for search result clustering. + (Brad Giaccio, Dawid Weiss, Stanislaw Osinski, rmuir, koji) + +* SOLR-2210: Add icu-based tokenizer and filters to contrib/analysis-extras (rmuir) + +* SOLR-1336: Add SmartChinese (word segmentation for Simplified Chinese) + tokenizer and filters to contrib/analysis-extras (rmuir) + +* SOLR-2211,LUCENE-2763: Added UAX29URLEmailTokenizerFactory, which implements + UAX#29, a unicode algorithm with good results for most languages, as well as + URL and E-mail tokenization according to the relevant RFCs. + (Tom Burton-West via rmuir) + +* SOLR-2237: Added StempelPolishStemFilterFactory to contrib/analysis-extras (rmuir) + +* SOLR-1525: allow DIH to refer to core properties (noble) + +* SOLR-1547: DIH TemplateTransformer copy objects more intelligently when the + template is a single variable (noble) + +* SOLR-1627: DIH VariableResolver should be fetched just in time (noble) + +* SOLR-1583: DIH Create DataSources that return InputStream (noble) + +* SOLR-1358: Integration of Tika and DataImportHandler (Akshay Ukey, noble) + +* SOLR-1654: TikaEntityProcessor example added DIHExample + (Akshay Ukey via noble) + +* SOLR-1678: Move onError handling to DIH framework (noble) + +* SOLR-1352: Multi-threaded implementation of DIH (noble) + +* SOLR-1721: Add explicit option to run DataImportHandler in synchronous mode + (Alexey Serba via noble) + +* SOLR-1737: Added FieldStreamDataSource (noble) + + +Optimizations +---------------------- + +* SOLR-1679: Don't build up string messages in SolrCore.execute unless they + are necessary for the current log level. + (Fuad Efendi and hossman) + +* SOLR-1874: Optimize PatternReplaceFilter for better performance. (rmuir, uschindler) + +* SOLR-1968: speed up initial filter cache population for facet.method=enum and + also big terms for multi-valued facet.method=fc. The resulting speedup + for the first facet request is anywhere from 30% to 32x, depending on how many + terms are in the field and how many documents match per term. (yonik) + +* SOLR-2089: Speed up UnInvertedField faceting (facet.method=fc for + multi-valued fields) when facet.limit is both high, and a high enough + percentage of the number of unique terms in the field. Extreme cases + yield speedups over 3x. (yonik) + +* SOLR-2046: add common functions to scripts-util. (koji) + +* SOLR-1684: Switch clustering component to use the + SolrIndexSearcher.doc(int, Set<String>) method b/c it can use the document + cache (gsingers) + +* SOLR-2200: Improve the performance of DataImportHandler for large + delta-import updates. (Mark Waddle via rmuir) + +Bug Fixes +---------------------- +* SOLR-1769: Solr 1.4 Replication - Repeater throwing NullPointerException (Jörgen Rydenius via noble) + +* SOLR-1432: Make the new ValueSource.getValues(context,reader) delegate + to the original ValueSource.getValues(reader) so custom sources + will work. (yonik) + +* SOLR-1572: FastLRUCache correctly implemented the LRU policy only + for the first 2B accesses. (yonik) + +* SOLR-1582: copyField was ignored for BinaryField types (gsingers) + +* SOLR-1563: Binary fields, including trie-based numeric fields, caused null + pointer exceptions in the luke request handler. (yonik) + +* SOLR-1577: The example solrconfig.xml defaulted to a solr data dir + relative to the current working directory, even if a different solr home + was being used. The new behavior changes the default to a zero length + string, which is treated the same as if no dataDir had been specified, + hence the "data" directory under the solr home will be used. (yonik) + +* SOLR-1584: SolrJ - SolrQuery.setIncludeScore() incorrectly added + fl=score to the parameter list instead of appending score to the + existing field list. (yonik) + +* SOLR-1580: Solr Configuration ignores 'mergeFactor' parameter, always + uses Lucene default. (Lance Norskog via Mark Miller) + +* SOLR-1593: ReverseWildcardFilter didn't work for surrogate pairs + (i.e. code points outside of the BMP), resulting in incorrect + matching. This change requires reindexing for any content with + such characters. (Robert Muir, yonik) + +* SOLR-1596: A rollback operation followed by the shutdown of Solr + or the close of a core resulted in a warning: + "SEVERE: SolrIndexWriter was not closed prior to finalize()" although + there were no other consequences. (yonik) + +* SOLR-1595: StreamingUpdateSolrServer used the platform default character + set when streaming updates, rather than using UTF-8 as the HTTP headers + indicated, leading to an encoding mismatch. (hossman, yonik) + +* SOLR-1587: A distributed search request with fl=score, didn't match + the behavior of a non-distributed request since it only returned + the id,score fields instead of all fields in addition to score. (yonik) + +* SOLR-1601: Schema browser does not indicate presence of charFilter. (koji) + +* SOLR-1615: Backslash escaping did not work in quoted strings + for local param arguments. (Wojtek Piaseczny, yonik) + +* SOLR-1628: log contains incorrect number of adds and deletes. + (Thijs Vonk via yonik) + +* SOLR-343: Date faceting now respects facet.mincount limiting + (Uri Boness, Raiko Eckstein via hossman) + +* SOLR-1624: Highlighter only highlights values from the first field value + in a multivalued field when term positions (term vectors) are stored. + (Chris Harris via yonik) + +* SOLR-1635: Fixed error message when numeric values can't be parsed by + DOMUtils - notably for plugin init params in solrconfig.xml. + (hossman) + +* SOLR-1651: Fixed Incorrect dataimport handler package name in SolrResourceLoader + (Akshay Ukey via shalin) + +* SOLR-1660: CapitalizationFilter crashes if you use the maxWordCountOption + (Robert Muir via shalin) + +* SOLR-1667: PatternTokenizer does not reset attributes such as positionIncrementGap + (Robert Muir via shalin) + +* SOLR-1711: SolrJ - StreamingUpdateSolrServer had a race condition that + could halt the streaming of documents. The original patch to fix this + (never officially released) introduced another hanging bug due to + connections not being released. + (Attila Babo, Erik Hetzner, Johannes Tuchscherer via yonik) + +* SOLR-1748, SOLR-1747, SOLR-1746, SOLR-1745, SOLR-1744: Streams and Readers + retrieved from ContentStreams are not closed in various places, resulting + in file descriptor leaks. + (Christoff Brill, Mark Miller) + +* SOLR-1753: StatsComponent throws NPE when getting statistics for facets in distributed search + (Janne Majaranta via koji) + +* SOLR-1736:In the slave , If 'mov'ing file does not succeed , copy the file (noble) + +* SOLR-1579: Fixes to XML escaping in stats.jsp + (David Bowen and hossman) + +* SOLR-1777: fieldTypes with sortMissingLast=true or sortMissingFirst=true can + result in incorrectly sorted results. (yonik) + +* SOLR-1798: Small memory leak (~100 bytes) in fastLRUCache for every + commit. (yonik) + +* SOLR-1823: Fixed XMLResponseWriter (via XMLWriter) so it no longer throws + a ClassCastException when a Map containing a non-String key is used. + (Frank Wesemann, hossman) + +* SOLR-1797: fix ConcurrentModificationException and potential memory + leaks in ResourceLoader. (yonik) + +* SOLR-1850: change KeepWordFilter so a new word set is not created for + each instance (John Wang via yonik) + +* SOLR-1706: fixed WordDelimiterFilter for certain combinations of options + where it would output incorrect tokens. (Robert Muir, Chris Male) + +* SOLR-1936: The JSON response format needed to escape unicode code point + U+2028 - 'LINE SEPARATOR' (Robert Hofstra, yonik) + +* SOLR-1914: Change the JSON response format to output float/double + values of NaN,Infinity,-Infinity as strings. (yonik) + +* SOLR-1948: PatternTokenizerFactory should use parent's args (koji) + +* SOLR-1870: Indexing documents using the 'javabin' format no longer + fails with a ClassCastException whenSolrInputDocuments contain field + values which are Collections or other classes that implement + Iterable. (noble, hossman) + +* SOLR-1981: Solr will now fail correctly if solr.xml attempts to + specify multiple cores that have the same name (hossman) + +* SOLR-1791: Fix messed up core names on admin gui (yonik via koji) + +* SOLR-1995: Change date format from "hour in am/pm" to "hour in day" + in CoreContainer and SnapShooter. (Hayato Ito, koji) + +* SOLR-2008: avoid possible RejectedExecutionException w/autoCommit + by making SolreCore close the UpdateHandler before closing the + SearchExecutor. (NarasimhaRaju, hossman) + +* SOLR-2036: Avoid expensive fieldCache ram estimation for the + admin stats page. (yonik) + +* SOLR-2047: ReplicationHandler should accept bool type for enable flag. (koji) + +* SOLR-1630: Fix spell checking collation issue related to token positions (rmuir, gsingers) + +* SOLR-2100: The replication handler backup command didn't save the commit + point and hence could fail when a newer commit caused the older commit point + to be removed before it was finished being copied. This did not affect + normal master/slave replication. (Peter Sturge via yonik) + +* SOLR-2114: Fixed parsing error in hsin function. The function signature has changed slightly. (gsingers) + +* SOLR-2083: SpellCheckComponent misreports suggestions when distributed (James Dyer via gsingers) + +* SOLR-2111: Change exception handling in distributed faceting to work more + like non-distributed faceting, change facet_counts/exception from a String + to a List<String> to enable listing all exceptions that happened, and + prevent an exception in one facet command from affecting another + facet command. (yonik) + +* SOLR-2110: Remove the restriction on names for local params + substitution/dereferencing. Properly encode local params in + distributed faceting. (yonik) + +* SOLR-2135: Fix behavior of ConcurrentLRUCache when asking for + getLatestAccessedItems(0) or getOldestAccessedItems(0). + (David Smiley via hossman) + +* SOLR-2148: Highlighter doesn't support q.alt. (koji) + +* SOLR-2180: It was possible for EmbeddedSolrServer to leave searchers + open if a request threw an exception. (yonik) + +* SOLR-2173: Suggester should always rebuild Lookup data if Lookup.load fails. (ab) + +* SOLR-2081: BaseResponseWriter.isStreamingDocs causes + SingleResponseWriter.end to be called 2x + (Chris A. Mattmann via hossman) + +* SOLR-2219: The init() method of every SolrRequestHandler was being + called twice. (ambikeshwar singh and hossman) + +* SOLR-2285: duplicate SolrEventListeners no longer created (hossman) + +* SOLR-1993: fix String cast assumption in JavaBinCodec - specific + addresses "commitWithin" option on Update requests. + (noble, hossman, and Maxim Valyanskiy) + +* SOLR-2261: fix velocity template layout.vm that referred to an older + version of jquery. (Eric Pugh via rmuir) + +* SOLR-2307: fix bug in PHPSerializedResponseWriter (wt=phps) when + dealing with SolrDocumentList objects -- ie: sharded queries. + (Antonio Verni via hossman) + +* SOLR-2127: Fixed serialization of default core and indentation of solr.xml when serializing. + (Ephraim Ofir, Mark Miller) + +* SOLR-2320: Fixed ReplicationHandler detail reporting for masters + (hossman) + +* SOLR-482: Provide more exception handling in CSVLoader (gsingers) + +* SOLR-1283: HTMLStripCharFilter sometimes threw a "Mark Invalid" exception. + (Julien Coloos, hossman, yonik) + +* SOLR-2085: Improve SolrJ behavior when FacetComponent comes before + QueryComponent (Tomas Salfischberger via hossman) + +* SOLR-1940: Fix SolrDispatchFilter behavior when Content-Type is + unknown (Lance Norskog and hossman) + +* SOLR-1983: snappuller fails when modifiedConfFiles is not empty and + full copy of index is needed. (Alexander Kanarsky via yonik) + +* SOLR-2156: SnapPuller fails to clean Old Index Directories on Full Copy + (Jayendra Patil via yonik) + +* SOLR-96: Fix XML parsing in XMLUpdateRequestHandler and + DocumentAnalysisRequestHandler to respect charset from XML file and only + use HTTP header's "Content-Type" as a "hint". (uschindler) + +* SOLR-2339: Fix sorting to explicitly generate an error if you + attempt to sort on a multiValued field. (hossman) + +* SOLR-2348: Fix field types to explicitly generate an error if you + attempt to get a ValueSource for a multiValued field. (hossman) + +* SOLR-2380: Distributed faceting could miss values when facet.sort=index + and when facet.offset was greater than 0. (yonik) + +* SOLR-1656: XIncludes and other HREFs in XML files loaded by ResourceLoader + are fixed to be resolved using the URI standard (RFC 2396). The system + identifier is no longer a plain filename with path, it gets initialized + using a custom URI scheme "solrres:". This scheme is resolved using a + EntityResolver that utilizes ResourceLoader + (org.apache.solr.common.util.SystemIdResolver). This makes all relative + pathes in Solr's config files behave like expected. This change + introduces some backwards breaks in the API: Some config classes + (Config, SolrConfig, IndexSchema) were changed to take + org.xml.sax.InputSource instead of InputStream. There may also be some + backwards breaks in existing config files, it is recommended to check + your config files / XSLTs and replace all XIncludes/HREFs that were + hacked to use absolute paths to use relative ones. (uschindler) + +* SOLR-309: Fix FieldType so setting an analyzer on a FieldType that + doesn't expect it will generate an error. Practically speaking this + means that Solr will now correctly generate an error on + initialization if the schema.xml contains an analyzer configuration + for a fieldType that does not use TextField. (hossman) + +* SOLR-2192: StreamingUpdateSolrServer.blockUntilFinished was not + thread safe and could throw an exception. (yonik) + +* SOLR-1692: Fix bug in clustering component relating to carrot.produceSummary + option (gsingers) + +* SOLR-1756: The date.format setting for extraction request handler causes + ClassCastException when enabled and the config code that parses this setting + does not properly use the same iterator instance. + (Christoph Brill, Mark Miller) + +* SOLR-1638: Fixed NullPointerException during DIH import if uniqueKey is not + specified in schema (Akshay Ukey via shalin) + +* SOLR-1639: Fixed misleading error message when dataimport.properties is not + writable (shalin) + +* SOLR-1598: DIH: Reader used in PlainTextEntityProcessor is not explicitly + closed (Sascha Szott via noble) + +* SOLR-1759: DIH: $skipDoc was not working correctly + (Gian Marco Tagliani via noble) + +* SOLR-1762: DIH: DateFormatTransformer does not work correctly with + non-default locale dates (tommy chheng via noble) + +* SOLR-1757: DIH multithreading sometimes throws NPE (noble) + +* SOLR-1766: DIH with threads enabled doesn't respond to the abort command + (Michael Henson via noble) + +* SOLR-1767: dataimporter.functions.escapeSql() does not escape backslash + character (Sean Timm via noble) + +* SOLR-1811: formatDate should use the current NOW value always + (Sean Timm via noble) + +* SOLR-1794: Dataimport of CLOB fields fails when getCharacterStream() is + defined in a superclass. (Gunnar Gauslaa Bergem via rmuir) + +* SOLR-2057: DataImportHandler never calls UpdateRequestProcessor.finish() + (Drew Farris via koji) + +* SOLR-1973: Empty fields in XML update messages confuse DataImportHandler. + (koji) + +* SOLR-2221: Use StrUtils.parseBool() to get values of boolean options in DIH. + true/on/yes (for TRUE) and false/off/no (for FALSE) can be used for + sub-options (debug, verbose, synchronous, commit, clean, optimize) for + full/delta-import commands. (koji) + +* SOLR-2310: DIH: getTimeElapsedSince() returns incorrect hour value when + the elapse is over 60 hours (tom liu via koji) + +* SOLR-2252: DIH: When a child entity in nested entities is rootEntity="true", + delta-import doesn't work. (koji) + +* SOLR-2330: solrconfig.xml files in example-DIH are broken. (Matt Parker, koji) + +* SOLR-1191: resolve DataImportHandler deltaQuery column against pk when pk + has a prefix (e.g. pk="book.id" deltaQuery="select id from ..."). More + useful error reporting when no match found (previously failed with a + NullPointerException in log and no clear user feedback). (gthb via yonik) + +* SOLR-2116: Fix TikaConfig classloader bug in TikaEntityProcessor + (Martijn van Groningen via hossman) + +Other Changes +---------------------- + +* SOLR-1602: Refactor SOLR package structure to include o.a.solr.response + and move QueryResponseWriters in there + (Chris A. Mattmann, ryan, hoss) + +* SOLR-1516: Addition of an abstract BaseResponseWriter class to simplify the + development of QueryResponseWriter implementations. + (Chris A. Mattmann via noble) + +* SOLR-1592: Refactor XMLWriter startTag to allow arbitrary attributes to be written + (Chris A. Mattmann via noble) + +* SOLR-1561: Added Lucene 2.9.1 spatial contrib jar to lib. (gsingers) + +* SOLR-1570: Log warnings if uniqueKey is multi-valued or not stored (hossman, shalin) + +* SOLR-1558: QueryElevationComponent only works if the uniqueKey field is + implemented using StrField. In previous versions of Solr no warning or + error would be generated if you attempted to use QueryElevationComponent, + it would just fail in unexpected ways. This has been changed so that it + will fail with a clear error message on initialization. (hossman) + +* SOLR-1611: Added Lucene 2.9.1 collation contrib jar to lib (shalin) + +* SOLR-1608: Extract base class from TestDistributedSearch to make + it easy to write test cases for other distributed components. (shalin) + +* Upgraded to Lucene 2.9-dev r888785 (shalin) + +* SOLR-1610: Generify SolrCache (Jason Rutherglen via shalin) + +* SOLR-1637: Remove ALIAS command + +* SOLR-1662: Added Javadocs in BufferedTokenStream and fixed incorrect cloning + in TestBufferedTokenStream (Robert Muir, Uwe Schindler via shalin) + +* SOLR-1674: Improve analysis tests and cut over to new TokenStream API. + (Robert Muir via Mark Miller) + +* SOLR-1661: Remove adminCore from CoreContainer . removed deprecated methods setAdminCore(), getAdminCore() (noble) + +* SOLR-1704: Google collections moved from clustering to core (noble) + +* SOLR-1268: Add Lucene 2.9-dev r888785 FastVectorHighlighter contrib jar to lib. (koji) + +* SOLR-1538: Reordering of object allocations in ConcurrentLRUCache to eliminate + (an extremely small) potential for deadlock. + (gabriele renzi via hossman) + +* SOLR-1588: Removed some very old dead code. + (Chris A. Mattmann via hossman) + +* SOLR-1696 : Deprecate old <highlighting> syntax and move configuration to HighlightComponent (noble) + +* SOLR-1727: SolrEventListener should extend NamedListInitializedPlugin (noble) + +* SOLR-1771: Improved error message when StringIndex cannot be initialized + for a function query (hossman) + +* SOLR-1695: Improved error messages when adding a document that does not + contain exactly one value for the uniqueKey field (hossman) + +* SOLR-1776: DismaxQParser and ExtendedDismaxQParser now use the schema.xml + "defaultSearchField" as the default value for the "qf" param instead of failing + with an error when "qf" is not specified. (hossman) + +* SOLR-1851: luceneAutoCommit no longer has any effect - it has been remove (Mark Miller) + +* SOLR-1865: SolrResourceLoader.getLines ignores Byte Order Markers (BOMs) at the + beginning of input files, these are often created by editors such as Windows + Notepad. (rmuir, hossman) + +* SOLR-1938: ElisionFilterFactory will use a default set of French contractions + if you do not supply a custom articles file. (rmuir) + +* SOLR-2003: SolrResourceLoader will report any encoding errors, rather than + silently using replacement characters for invalid inputs (blargy via rmuir) + +* SOLR-1804: Google collections updated to Google Guava (which is a superset of collections and contains bug fixes) (gsingers) + +* SOLR-2034: Switch to JavaBin codec version 2. Strings are now serialized + as the number of UTF-8 bytes, followed by the bytes in UTF-8. Previously + Strings were serialized as the number of UTF-16 chars, followed by the + bytes in Modified UTF-8. (hossman, yonik, rmuir) + +* SOLR-2013: Add mapping-FoldToASCII.txt to example conf directory. + (Steven Rowe via koji) + +* SOLR-2213: Upgrade to jQuery 1.4.3 (Erick Erickson via ryan) + +* SOLR-1826: Add unit tests for highlighting with termOffsets=true + and overlapping tokens. (Stefan Oestreicher via rmuir) + +* SOLR-2340: Add version infos to message in JavaBinCodec when throwing + exception. (koji) + +* SOLR-2350: Since Solr no longer requires XML files to be in UTF-8 + (see SOLR-96) SimplePostTool (aka: post.jar) has been improved to + work with files of any mime-type or charset. (hossman) + +* SOLR-2365: Move DIH jars out of solr.war (David Smiley via yonik) + +* SOLR-2381: Include a patched version of Jetty (6.1.26 + JETTY-1340) + to fix problematic UTF-8 handling for supplementary characters. + (Bernd Fehling, uschindler, yonik, rmuir) + +* SOLR-2391: The preferred Content-Type for XML was changed to + application/xml. XMLResponseWriter now only delivers using this + type; updating documents and analyzing documents is still supported + using text/xml as Content-Type, too. If you have clients that are + hardcoded on text/xml as Content-Type, you have to change them. + (uschindler, rmuir) + +* SOLR-2414: All ResponseWriters now use only ServletOutputStreams + and wrap their own Writer around it when serializing. This fixes + the bug in PHPSerializedResponseWriter that produced wrong string + length if the servlet container had a broken UTF-8 encoding that was + in fact CESU-8 (see SOLR-1091). The system property to enable the + CESU-8 byte counting in PHPSerializesResponseWriters for broken + servlet containers was therefore removed and is now ignored if set. + Output is always UTF-8. (uschindler, yonik, rmuir) + +* SOLR-141: Errors and Exceptions are formated by ResponseWriter. + (Mike Sokolov, Rich Cariens, Daniel Naber, ryan) + +* SOLR-1902: Upgraded to Tika 0.8 and changed deprecated parse call + +* SOLR-1813: Add ICU4j to contrib/extraction libs and add tests for Arabic + extraction (Robert Muir via gsingers) + +* SOLR-1821: Fix TimeZone-dependent test failure in TestEvaluatorBag. + (Chris Male via rmuir) + +* SOLR-2367: Reduced noise in test output by ensuring the properties file + can be written. (Gunnlaugur Thor Briem via rmuir) + +Build +---------------------- + +* SOLR-1522: Automated release signing process. (gsingers) + +* SOLR-1891: Make lucene-jars-to-solr fail if copying any of the jars fails, and + update clean to remove the jars in that directory (Mark Miller) + +* LUCENE-2466: Commons-Codec was upgraded from 1.3 to 1.4. (rmuir) + +* SOLR-2042: Fixed some Maven deps (Drew Farris via gsingers) + +* LUCENE-2657: Switch from using Maven POM templates to full POMs when + generating Maven artifacts (Steven Rowe) + +Documentation +---------------------- + +* SOLR-1590: Javadoc for XMLWriter#startTag + (Chris A. Mattmann via hossman) + +* SOLR-1792: Documented peculiar behavior of TestHarness.LocalRequestFactory + (hossman) + +================== Release 1.4.1 ================== +Release Date: See http://lucene.apache.org/solr for the official release date. + +Upgrading from Solr 1.4 +----------------------- + +This is a bug fix release - no changes are required when upgrading from Solr 1.4. +However, a reindex is needed for some of the analysis fixes to take effect. + +Versions of Major Components +---------------------------- +Apache Lucene 2.9.3 +Apache Tika 0.4 +Carrot2 3.1.0 + +Lucene Information +---------------- + +Since Solr is built on top of Lucene, many people add customizations to Solr +that are dependent on Lucene. Please see http://lucene.apache.org/java/2_9_3/, +especially http://lucene.apache.org/java/2_9_3/changes/Changes.html for more +information on the version of Lucene used in Solr. + +Bug Fixes +---------------------- + +* SOLR-1934: Upgrade to Apache Lucene 2.9.3 to obtain several bug + fixes from the previous 2.9.1. See the Lucene 2.9.3 release notes + for details. (hossman, Mark Miller) + +* SOLR-1432: Make the new ValueSource.getValues(context,reader) delegate + to the original ValueSource.getValues(reader) so custom sources + will work. (yonik) + +* SOLR-1572: FastLRUCache correctly implemented the LRU policy only + for the first 2B accesses. (yonik) + +* SOLR-1595: StreamingUpdateSolrServer used the platform default character + set when streaming updates, rather than using UTF-8 as the HTTP headers + indicated, leading to an encoding mismatch. (hossman, yonik) + +* SOLR-1660: CapitalizationFilter crashes if you use the maxWordCountOption + (Robert Muir via shalin) + +* SOLR-1662: Added Javadocs in BufferedTokenStream and fixed incorrect cloning + in TestBufferedTokenStream (Robert Muir, Uwe Schindler via shalin) + +* SOLR-1711: SolrJ - StreamingUpdateSolrServer had a race condition that + could halt the streaming of documents. The original patch to fix this + (never officially released) introduced another hanging bug due to + connections not being released. (Attila Babo, Erik Hetzner via yonik) + +* SOLR-1748, SOLR-1747, SOLR-1746, SOLR-1745, SOLR-1744: Streams and Readers + retrieved from ContentStreams are not closed in various places, resulting + in file descriptor leaks. + (Christoff Brill, Mark Miller) + +* SOLR-1580: Solr Configuration ignores 'mergeFactor' parameter, always + uses Lucene default. (Lance Norskog via Mark Miller) + +* SOLR-1777: fieldTypes with sortMissingLast=true or sortMissingFirst=true can + result in incorrectly sorted results. (yonik) + +* SOLR-1797: fix ConcurrentModificationException and potential memory + leaks in ResourceLoader. (yonik) + +* SOLR-1798: Small memory leak (~100 bytes) in fastLRUCache for every + commit. (yonik) + +* SOLR-1522: Show proper message if <script> tag is missing for DIH + ScriptTransformer (noble) + +* SOLR-1538: Reordering of object allocations in ConcurrentLRUCache to eliminate + (an extremely small) potential for deadlock. + (gabriele renzi via hossman) + +* SOLR-1558: QueryElevationComponent only works if the uniqueKey field is + implemented using StrField. In previous versions of Solr no warning or + error would be generated if you attempted to use QueryElevationComponent, + it would just fail in unexpected ways. This has been changed so that it + will fail with a clear error message on initialization. (hossman) + +* SOLR-1563: Binary fields, including trie-based numeric fields, caused null + pointer exceptions in the luke request handler. (yonik) + +* SOLR-1579: Fixes to XML escaping in stats.jsp + (David Bowen and hossman) + +* SOLR-1582: copyField was ignored for BinaryField types (gsingers) + +* SOLR-1596: A rollback operation followed by the shutdown of Solr + or the close of a core resulted in a warning: + "SEVERE: SolrIndexWriter was not closed prior to finalize()" although + there were no other consequences. (yonik) + +* SOLR-1651: Fixed Incorrect dataimport handler package name in SolrResourceLoader + (Akshay Ukey via shalin) + +* SOLR-1936: The JSON response format needed to escape unicode code point + U+2028 - 'LINE SEPARATOR' (Robert Hofstra, yonik) + +* SOLR-1852: Fix WordDelimiterFilterFactory bug where position increments + were not being applied properly to subwords. (Peter Wolanin via Robert Muir) + +* SOLR-1706: fixed WordDelimiterFilter for certain combinations of options + where it would output incorrect tokens. (Robert Muir, Chris Male) + +* SOLR-1948: PatternTokenizerFactory should use parent's args (koji) + +* SOLR-1870: Indexing documents using the 'javabin' format no longer + fails with a ClassCastException whenSolrInputDocuments contain field + values which are Collections or other classes that implement + Iterable. (noble, hossman) + +* SOLR-1769 Solr 1.4 Replication - Repeater throwing NullPointerException (noble) + +================== Release 1.4.0 ================== +Release Date: See http://lucene.apache.org/solr for the official release date. + +Upgrading from Solr 1.3 +----------------------- + +There is a new default faceting algorithm for multiVaued fields that should be +faster for most cases. One can revert to the previous algorithm (which has +also been improved somewhat) by adding facet.method=enum to the request. + +Searching and sorting is now done on a per-segment basis, meaning that +the FieldCache entries used for sorting and for function queries are +created and used per-segment and can be reused for segments that don't +change between index updates. While generally beneficial, this can lead +to increased memory usage over 1.3 in certain scenarios: + 1) A single valued field that was used for both sorting and faceting +in 1.3 would have used the same top level FieldCache entry. In 1.4, +sorting will use entries at the segment level while faceting will still +use entries at the top reader level, leading to increased memory usage. + 2) Certain function queries such as ord() and rord() require a top level +FieldCache instance and can thus lead to increased memory usage. Consider +replacing ord() and rord() with alternatives, such as function queries +based on ms() for date boosting. + +If you use custom Tokenizer or TokenFilter components in a chain specified in +schema.xml, they must support reusability. If your Tokenizer or TokenFilter +maintains state, it should implement reset(). If your TokenFilteFactory does +not return a subclass of TokenFilter, then it should implement reset() and call +reset() on its input TokenStream. TokenizerFactory implementations must +now return a Tokenizer rather than a TokenStream. + +New users of Solr 1.4 will have omitTermFreqAndPositions enabled for non-text +indexed fields by default, which avoids indexing term frequency, positions, and +payloads, making the index smaller and faster. If you are upgrading from an +earlier Solr release and want to enable omitTermFreqAndPositions by default, +change the schema version from 1.1 to 1.2 in schema.xml. Remove any existing +index and restart Solr to ensure that omitTermFreqAndPositions completely takes +affect. + +The default QParserPlugin used by the QueryComponent for parsing the "q" param +has been changed, to remove support for the deprecated use of ";" as a separator +between the query string and the sort options when no "sort" param was used. +Users who wish to continue using the semi-colon based method of specifying the +sort options should explicitly set the defType param to "lucenePlusSort" on all +requests. (The simplest way to do this is by specifying it as a default param +for your request handlers in solrconfig.xml, see the example solrconfig.xml for +sample syntax.) + +If spellcheck.extendedResults=true, the response format for suggestions +has changed, see SOLR-1071. + +Use of the "charset" option when configuring the following Analysis +Factories has been deprecated and will cause a warning to be logged. +In future versions of Solr attempting to use this option will cause an +error. See SOLR-1410 for more information. + - GreekLowerCaseFilterFactory + - RussianStemFilterFactory + - RussianLowerCaseFilterFactory + - RussianLetterTokenizerFactory + +DIH: Evaluator API has been changed in a non back-compatible way. Users who +have developed custom Evaluators will need to change their code according to +the new API for it to work. See SOLR-996 for details. + +DIH: The formatDate evaluator's syntax has been changed. The new syntax is +formatDate(<variable>, '<format_string>'). For example, +formatDate(x.date, 'yyyy-MM-dd'). In the old syntax, the date string was +written without a single-quotes. The old syntax has been deprecated and will +be removed in 1.5, until then, using the old syntax will log a warning. + +DIH: The Context API has been changed in a non back-compatible way. In +particular, the Context.currentProcess() method now returns a String +describing the type of the current import process instead of an int. +Similarily, the public constants in Context viz. FULL_DUMP, DELTA_DUMP and +FIND_DELTA are changed to a String type. See SOLR-969 for details. + +DIH: The EntityProcessor API has been simplified by moving logic for applying +transformers and handling multi-row outputs from Transformers into an +EntityProcessorWrapper class. The EntityProcessor#destroy is now called once +per parent-row at the end of row (end of data). A new method +EntityProcessor#close is added which is called at the end of import. + +DIH: In Solr 1.3, if the last_index_time was not available (first import) and +a delta-import was requested, a full-import was run instead. This is no longer +the case. In Solr 1.4 delta import is run with last_index_time as the epoch +date (January 1, 1970, 00:00:00 GMT) if last_index_time is not available. + +Versions of Major Components +---------------------------- +Apache Lucene 2.9.1 (r832363 on 2.9 branch) +Apache Tika 0.4 +Carrot2 3.1.0 + +Lucene Information +---------------- + +Since Solr is built on top of Lucene, many people add customizations to Solr +that are dependent on Lucene. Please see http://lucene.apache.org/java/2_9_0/, +especially http://lucene.apache.org/java/2_9_0/changes/Changes.html for more +information on the version of Lucene used in Solr. + +Detailed Change List +---------------------- + +New Features +---------------------- + 1. SOLR-560: Use SLF4J logging API rather then JDK logging. The packaged .war file is + shipped with a JDK logging implementation, so logging configuration for the .war should + be identical to solr 1.3. However, if you are using the .jar file, you can select + which logging implementation to use by dropping a different binding. + See: http://www.slf4j.org/ (ryan) + + 2. SOLR-617: Allow configurable index deletion policy and provide a default implementation which + allows deletion of commit points on various criteria such as number of commits, age of commit + point and optimized status. + See http://lucene.apache.org/java/2_3_2/api/org/apache/lucene/index/IndexDeletionPolicy.html + (yonik, Noble Paul, Akshay Ukey via shalin) + + 3. SOLR-658: Allow Solr to load index from arbitrary directory in dataDir + (Noble Paul, Akshay Ukey via shalin) + + 4. SOLR-793: Add 'commitWithin' argument to the update add command. This behaves + similar to the global autoCommit maxTime argument except that it is set for + each request. (ryan) + + 5. SOLR-670: Add support for rollbacks in UpdateHandler. This allows user to rollback all changes + since the last commit. (Noble Paul, koji via shalin) + + 6. SOLR-813: Adding DoubleMetaphone Filter and Factory. Similar to the PhoneticFilter, + but this uses DoubleMetaphone specific calls (including alternate encoding) + (Todd Feak via ryan) + + 7. SOLR-680: Add StatsComponent. This gets simple statistics on matched numeric fields, + including: min, max, mean, median, stddev. (koji, ryan) + + - SOLR-1380: Added support for multi-valued fields (Harish Agarwal via gsingers) + + 8. SOLR-561: Added Replication implemented in Java as a request handler. Supports index replication + as well as configuration replication and exposes detailed statistics and progress information + on the Admin page. Works on all platforms. (Noble Paul, yonik, Akshay Ukey, shalin) + + 9. SOLR-746: Added "omitHeader" request parameter to omit the header from the response. + (Noble Paul via shalin) + +10. SOLR-651: Added TermVectorComponent for serving up term vector information, plus IDF. + See http://wiki.apache.org/solr/TermVectorComponent (gsingers, Vaijanath N. Rao, Noble Paul) + +12. SOLR-795: SpellCheckComponent supports building indices on optimize if configured in solrconfig.xml + (Jason Rennie, shalin) + +13. SOLR-667: A LRU cache implementation based upon ConcurrentHashMap and other techniques to reduce + contention and synchronization overhead, to utilize multiple CPU cores more effectively. + (Fuad Efendi, Noble Paul, yonik via shalin) + +14. SOLR-465: Add configurable DirectoryProvider so that alternate Directory + implementations can be specified via solrconfig.xml. The default + DirectoryProvider will use NIOFSDirectory for better concurrency + on non Windows platforms. (Mark Miller, TJ Laurenzo via yonik) + +15. SOLR-822: Add CharFilter so that characters can be filtered (e.g. character normalization) + before Tokenizer/TokenFilters. (koji) + +16. SOLR-829: Allow slaves to request compressed files from master during replication + (Simon Collins, Noble Paul, Akshay Ukey via shalin) + +17. SOLR-877: Added TermsComponent for accessing Lucene's TermEnum capabilities. + Useful for auto suggest and possibly distributed search. Not distributed search compliant. (gsingers) + - Added mincount and maxcount options (Khee Chin via gsingers) + +18. SOLR-538: Add maxChars attribute for copyField function so that the length limit for destination + can be specified. + (Georgios Stamatis, Lars Kotthoff, Chris Harris via koji) + +19. SOLR-284: Added support for extracting content from binary documents like MS Word and PDF using Apache Tika. See also contrib/extraction/CHANGES.txt (Eric Pugh, Chris Harris, yonik, gsingers) + +20. SOLR-819: Added factories for Arabic support (gsingers) + +21. SOLR-781: Distributed search ability to sort field.facet values + lexicographically. facet.sort values "true" and "false" are + also deprecated and replaced with "count" and "lex". + (Lars Kotthoff via yonik) + +22. SOLR-821: Add support for replication to copy conf file to slave with a different name. This allows replication + of solrconfig.xml + (Noble Paul, Akshay Ukey via shalin) + +23. SOLR-911: Add support for multi-select faceting by allowing filters to be + tagged and facet commands to exclude certain filters. This patch also + added the ability to change the output key for facets in the response, and + optimized distributed faceting refinement by lowering parsing overhead and + by making requests and responses smaller. + +24. SOLR-876: WordDelimiterFilter now supports a splitOnNumerics + option, as well as a list of protected terms. + (Dan Rosher via hossman) + +25. SOLR-928: SolrDocument and SolrInputDocument now implement the Map<String,?> + interface. This should make plugging into other standard tools easier. (ryan) + +26. SOLR-847: Enhance the snappull command in ReplicationHandler to accept masterUrl. + (Noble Paul, Preetam Rao via shalin) + +27. SOLR-540: Add support for globbing in field names to highlight. + For example, hl.fl=*_text will highlight all fieldnames ending with + _text. (Lars Kotthoff via yonik) + +28. SOLR-906: Adding a StreamingUpdateSolrServer that writes update commands to + an open HTTP connection. If you are using solrj for bulk update requests + you should consider switching to this implementaion. However, note that + the error handling is not immediate as it is with the standard SolrServer. + (ryan) + +29. SOLR-865: Adding support for document updates in binary format and corresponding support in Solrj client. + (Noble Paul via shalin) + +30. SOLR-763: Add support for Lucene's PositionFilter (Mck SembWever via shalin) + +31. SOLR-966: Enhance the map() function query to take in an optional default value (Noble Paul, shalin) + +32. SOLR-820: Support replication on startup of master with new index. (Noble Paul, Akshay Ukey via shalin) + +33. SOLR-943: Make it possible to specify dataDir in solr.xml and accept the dataDir as a request parameter for + the CoreAdmin create command. (Noble Paul via shalin) + +34. SOLR-850: Addition of timeouts for distributed searching. Configurable through 'shard-socket-timeout' and + 'shard-connection-timeout' parameters in SearchHandler. (Patrick O'Leary via shalin) + +35. SOLR-799: Add support for hash based exact/near duplicate document + handling. (Mark Miller, yonik) + +36. SOLR-1026: Add protected words support to SnowballPorterFilterFactory (ehatcher) + +37. SOLR-739: Add support for OmitTf (Mark Miller via yonik) + +38. SOLR-1046: Nested query support for the function query parser + and lucene query parser (the latter existed as an undocumented + feature in 1.3) (yonik) + +39. SOLR-940: Add support for Lucene's Trie Range Queries by providing new FieldTypes in + schema for int, float, long, double and date. Single-valued Trie based + fields with a precisionStep will index multiple precisions and enable + faster range queries. (Uwe Schindler, yonik, shalin) + +40. SOLR-1038: Enhance CommonsHttpSolrServer to add docs in batch using an iterator API (Noble Paul via shalin) + +41. SOLR-844: A SolrServer implementation to front-end multiple solr servers and provides load balancing and failover + support (Noble Paul, Mark Miller, hossman via shalin) + +42. SOLR-939: ValueSourceRangeFilter/Query - filter based on values in a FieldCache entry or on any arbitrary function of field values. (yonik) + +43. SOLR-1095: Fixed performance problem in the StopFilterFactory and simplified code. Added tests as well. (gsingers) + +44. SOLR-1096: Introduced httpConnTimeout and httpReadTimeout in replication slave configuration to avoid stalled + replication. (Jeff Newburn, Noble Paul, shalin) + +45. SOLR-1115: <bool>on</bool> and <bool>yes</bool> work as expected in solrconfig.xml. (koji) + +46. SOLR-1099: A FieldAnalysisRequestHandler which provides the analysis functionality of the web admin page as + a service. The AnalysisRequestHandler is renamed to DocumentAnalysisRequestHandler which is enhanced with + query analysis and showMatch support. AnalysisRequestHandler is now deprecated. Support for both + FieldAnalysisRequestHandler and DocumentAnalysisRequestHandler is also provided in the Solrj client. + (Uri Boness, shalin) + +47. SOLR-1106: Made CoreAdminHandler Actions pluggable so that additional actions may be plugged in or the existing + ones can be overridden if needed. (Kay Kay, Noble Paul, shalin) + +48. SOLR-1124: Add a top() function query that causes its argument to + have its values derived from the top level IndexReader, even when + invoked from a sub-reader. top() is implicitly used for the + ord() and rord() functions. (yonik) + +49. SOLR-1110: Support sorting on trie fields with Distributed Search. (Mark Miller, Uwe Schindler via shalin) + +50. SOLR-1121: CoreAdminhandler should not need a core . This makes it possible to start a Solr server w/o a core .(noble) + +51. SOLR-769: Added support for clustering in contrib/clustering. See http://wiki.apache.org/solr/ClusteringComponent for more info. (gsingers, Stanislaw Osinski) + +52. SOLR-1175: disable/enable replication on master side. added two commands 'enableReplication' and 'disableReplication' (noble) + +53. SOLR-1179: DocSets can now be used as Lucene Filters via + DocSet.getTopFilter() (yonik) + +54. SOLR-1116: Add a Binary FieldType (noble) + +55. SOLR-1051: Support the merge of multiple indexes as a CoreAdmin and an update command (Ning Li via shalin) + +56. SOLR-1152: Snapshoot on ReplicationHandler should accept location as a request parameter (shalin) + +57. SOLR-1204: Enhance SpellingQueryConverter to handle UTF-8 instead of ASCII only. + Use the NMTOKEN syntax for matching field names. + (Michael Ludwig, shalin) + +58. SOLR-1189: Support providing username and password for basic HTTP authentication in Java replication + (Matthew Gregg, shalin) + +59. SOLR-243: Add configurable IndexReaderFactory so that alternate IndexReader implementations + can be specified via solrconfig.xml. Note that using a custom IndexReader may be incompatible + with ReplicationHandler (see comments in SOLR-1366). This should be treated as an experimental feature. + (Andrzej Bialecki, hossman, Mark Miller, John Wang) + +60. SOLR-1214: differentiate between solr home and instanceDir .deprecates the method SolrResourceLoader#locateInstanceDir() + and it is renamed to locateSolrHome (noble) + +61. SOLR-1216 : disambiguate the replication command names. 'snappull' becomes 'fetchindex' 'abortsnappull' becomes 'abortfetch' (noble) + +62. SOLR-1145: Add capability to specify an infoStream log file for the underlying Lucene IndexWriter in solrconfig.xml. + This is an advanced debug log file that can be used to aid developers in fixing IndexWriter bugs. See the commented + out example in the example solrconfig.xml under the indexDefaults section. + (Chris Harris, Mark Miller) + +63. SOLR-1256: Show the output of CharFilters in analysis.jsp. (koji) + +64. SOLR-1266: Added stemEnglishPossessive option (default=true) to WordDelimiterFilter + that allows disabling of english possessive stemming (removal of trailing 's from tokens) + (Robert Muir via yonik) + +65. SOLR-1237: firstSearcher and newSearcher can now be identified via the CommonParams.EVENT (evt) parameter + in a request. This allows a RequestHandler or SearchComponent to know when a newSearcher or firstSearcher + event happened. QuerySenderListender is the only implementation in Solr that implements this, but outside + implementations may wish to. See the AbstractSolrEventListener for a helper method. (gsingers) + +66. SOLR-1343: Added HTMLStripCharFilter and marked HTMLStripReader, HTMLStripWhitespaceTokenizerFactory and + HTMLStripStandardTokenizerFactory deprecated. To strip HTML tags, HTMLStripCharFilter can be used + with an arbitrary Tokenizer. (koji) + +67. SOLR-1275: Add expungeDeletes to DirectUpdateHandler2 (noble) + +68. SOLR-1372: Enhance FieldAnalysisRequestHandler to accept field value from content stream (ehatcher) + +69. SOLR-1370: Show the output of CharFilters in FieldAnalysisRequestHandler (koji) + +70. SOLR-1373: Add Filter query to admin/form.jsp + (Jason Rutherglen via hossman) + +71. SOLR-1368: Add ms() function query for getting milliseconds from dates and for + high precision date subtraction, add sub() for subtracting other arguments. + (yonik) + +72. SOLR-1156: Sort TermsComponent results by frequency (Matt Weber via yonik) + +73. SOLR-1335 : load core properties from a properties file (noble) + +74. SOLR-1385 : Add an 'enable' attribute to all plugins (noble) + +75. SOLR-1414 : implicit core properties are not set for single core (noble) + +76. SOLR-659 : Adds shards.start and shards.rows to distributed search + to allow more efficient bulk queries (those that retrieve many or all + documents). (Brian Whitman via yonik) + +77. SOLR-1321: Add better support for efficient wildcard handling (Andrzej Bialecki, Robert Muir, gsingers) + +78. SOLR-1326 : New interface PluginInfoInitialized for all types of plugin (noble) + +79. SOLR-1447 : Simple property injection. <mergePolicy> & <mergeScheduler> syntaxes are now deprecated + (Jason Rutherglen, noble) + +80. SOLR-908 : CommonGramsFilterFactory/CommonGramsQueryFilterFactory for + speeding up phrase queries containing common words by indexing + n-grams and using them at query time. + (Tom Burton-West, Jason Rutherglen via yonik) + +81. SOLR-1292: Add FieldCache introspection to stats.jsp and JMX Monitoring via + a new SolrFieldCacheMBean. (hossman) + +82. SOLR-1167: Solr Config now supports XInclude for XML engines that can support it. (Bryan Talbot via gsingers) + +83. SOLR-1478: Enable sort by Lucene docid. (ehatcher) + +84. SOLR-1449: Add <lib> elements to solrconfig.xml to specifying additional + classpath directories and regular expressions. (hossman via yonik) + +85. SOLR-1128: Added metadata output to extraction request handler "extract + only" option. (gsingers) + +86. SOLR-1274: Added text serialization output for extractOnly + (Peter Wolanin, gsingers) + +87. SOLR-768: DIH: Set last_index_time variable in full-import command. + (Wojtek Piaseczny, Noble Paul via shalin) + +88. SOLR-811: Allow a "deltaImportQuery" attribute in SqlEntityProcessor + which is used for delta imports instead of DataImportHandler manipulating + the SQL itself. (Noble Paul via shalin) + +89. SOLR-842: Better error handling in DataImportHandler with options to + abort, skip and continue imports. (Noble Paul, shalin) + +90. SOLR-833: DIH: A DataSource to read data from a field as a reader. This + can be used, for example, to read XMLs residing as CLOBs or BLOBs in + databases. (Noble Paul via shalin) + +91. SOLR-887: A DIH Transformer to strip HTML tags. (Ahmed Hammad via shalin) + +92. SOLR-886: DataImportHandler should rollback when an import fails or it is + aborted (shalin) + +93. SOLR-891: A DIH Transformer to read strings from Clob type. + (Noble Paul via shalin) + +94. SOLR-812: Configurable JDBC settings in JdbcDataSource including optimized + defaults for read only mode. (David Smiley, Glen Newton, shalin) + +95. SOLR-910: Add a few utility commands to the DIH admin page such as full + import, delta import, status, reload config. (Ahmed Hammad via shalin) + +96. SOLR-938: Add event listener API for DIH import start and end. + (Kay Kay, Noble Paul via shalin) + +97. SOLR-801: DIH: Add support for configurable pre-import and post-import + delete query per root-entity. (Noble Paul via shalin) + +98. SOLR-988: Add a new scope for session data stored in Context to store + objects across imports. (Noble Paul via shalin) + +99. SOLR-980: A PlainTextEntityProcessor which can read from any + DataSource<Reader> and output a String. + (Nathan Adams, Noble Paul via shalin) + +100.SOLR-1003: XPathEntityprocessor must allow slurping all text from a given + xml node and its children. (Noble Paul via shalin) + +101.SOLR-1001: Allow variables in various attributes of RegexTransformer, + HTMLStripTransformer and NumberFormatTransformer. + (Fergus McMenemie, Noble Paul, shalin) + +102.SOLR-989: DIH: Expose running statistics from the Context API. + (Noble Paul, shalin) + +103.SOLR-996: DIH: Expose Context to Evaluators. (Noble Paul, shalin) + +104.SOLR-783: DIH: Enhance delta-imports by maintaining separate + last_index_time for each entity. (Jon Baer, Noble Paul via shalin) + +105.SOLR-1033: Current entity's namespace is made available to all DIH + Transformers. This allows one to use an output field of TemplateTransformer + in other transformers, among other things. + (Fergus McMenemie, Noble Paul via shalin) + +106.SOLR-1066: New methods in DIH Context to expose Script details. + ScriptTransformer changed to read scripts through the new API methods. + (Noble Paul via shalin) + +107.SOLR-1062: A DIH LogTransformer which can log data in a given template + format. (Jon Baer, Noble Paul via shalin) + +108.SOLR-1065: A DIH ContentStreamDataSource which can accept HTTP POST data + in a content stream. This can be used to push data to Solr instead of + just pulling it from DB/Files/URLs. (Noble Paul via shalin) + +109.SOLR-1061: Improve DIH RegexTransformer to create multiple columns from + regex groups. (Noble Paul via shalin) + +110.SOLR-1059: Special DIH flags introduced for deleting documents by query or + id, skipping rows and stopping further transforms. Use $deleteDocById, + $deleteDocByQuery for deleting by id and query respectively. Use $skipRow + to skip the current row but continue with the document. Use $stopTransform + to stop further transformers. New methods are introduced in Context for + deleting by id and query. (Noble Paul, Fergus McMenemie, shalin) + +111.SOLR-1076: JdbcDataSource should resolve DIH variables in all its + configuration parameters. (shalin) + +112.SOLR-1055: Make DIH JdbcDataSource easily extensible by making the + createConnectionFactory method protected and return a + Callable<Connection> object. (Noble Paul, shalin) + +113.SOLR-1058: DIH: JdbcDataSource can lookup javax.sql.DataSource using JNDI. + Use a jndiName attribute to specify the location of the data source. + (Jason Shepherd, Noble Paul via shalin) + +114.SOLR-1083: A DIH Evaluator for escaping query characters. + (Noble Paul, shalin) + +115.SOLR-934: A MailEntityProcessor to enable indexing mails from + POP/IMAP sources into a solr index. (Preetam Rao, shalin) + +116.SOLR-1060: A DIH LineEntityProcessor which can stream lines of text from a + given file to be indexed directly or for processing with transformers and + child entities. + (Fergus McMenemie, Noble Paul, shalin) + +117.SOLR-1127: Add support for DIH field name to be templatized. + (Noble Paul, shalin) + +118.SOLR-1092: Added a new DIH command named 'import' which does not + automatically clean the index. This is useful and more appropriate when one + needs to import only some of the entities. + (Noble Paul via shalin) + +119.SOLR-1153: DIH 'deltaImportQuery' is honored on child entities as well + (noble) + +120.SOLR-1230: Enhanced dataimport.jsp to work with all DataImportHandler + request handler configurations, rather than just a hardcoded /dataimport + handler. (ehatcher) + +121.SOLR-1235: disallow period (.) in DIH entity names (noble) + +122.SOLR-1234: Multiple DIH does not work because all of them write to + dataimport.properties. Use the handler name as the properties file name + (noble) + +123.SOLR-1348: Support binary field type in convertType logic in DIH + JdbcDataSource (shalin) + +124.SOLR-1406: DIH: Make FileDataSource and FileListEntityProcessor to be more + extensible (Luke Forehand, shalin) + +125.SOLR-1437: DIH: XPathEntityProcessor can deal with xpath syntaxes such as + //tagname , /root//tagname (Fergus McMenemie via noble) + + +Optimizations +---------------------- + 1. SOLR-374: Use IndexReader.reopen to save resources by re-using parts of the + index that haven't changed. (Mark Miller via yonik) + + 2. SOLR-808: Write string keys in Maps as extern strings in the javabin format. (Noble Paul via shalin) + + 3. SOLR-475: New faceting method with better performance and smaller memory usage for + multi-valued fields with many unique values but relatively few values per document. + Controllable via the facet.method parameter - "fc" is the new default method and "enum" + is the original method. (yonik) + + 4. SOLR-970: Use an ArrayList in SolrPluginUtils.parseQueryStrings + since we know exactly how long the List will be in advance. + (Kay Kay via hossman) + + 5. SOLR-1002: Change SolrIndexSearcher to use insertWithOverflow + with reusable priority queue entries to reduce the amount of + generated garbage during searching. (Mark Miller via yonik) + + 6. SOLR-971: Replace StringBuffer with StringBuilder for instances that do not require thread-safety. + (Kay Kay via shalin) + + 7. SOLR-921: SolrResourceLoader must cache short class name vs fully qualified classname + (Noble Paul, hossman via shalin) + + 8. SOLR-973: CommonsHttpSolrServer writes the xml directly to the server. + (Noble Paul via shalin) + + 9. SOLR-1108: Remove un-needed synchronization in SolrCore constructor. + (Noble Paul via shalin) + +10. SOLR-1166: Speed up docset/filter generation by avoiding top-level + score() call and iterating over leaf readers with TermDocs. (yonik) + +11. SOLR-1169: SortedIntDocSet - a new small set implementation + that saves memory over HashDocSet, is faster to construct, + is ordered for easier implementation of skipTo, and is faster + in the general case. (yonik) + +12. SOLR-1165: Use Lucene Filters and pass them down to the Lucene + search methods to filter earlier and improve performance. (yonik) + +13. SOLR-1111: Use per-segment sorting to share fieldcache elements + across unchanged segments. This saves memory and reduces + commit times for incremental updates to the index. (yonik) + +14. SOLR-1188: Minor efficiency improvement in TermVectorComponent related to ignoring positions or offsets (gsingers) + +15. SOLR-1150: Load Documents for Highlighting one at a time rather than + all at once to avoid OOM with many large Documents. (Siddharth Gargate via Mark Miller) + +16. SOLR-1353: Implement and use reusable token streams for analysis. (Robert Muir, yonik) + +17. SOLR-1296: Enables setting IndexReader's termInfosIndexDivisor via a new attribute to StandardIndexReaderFactory. Enables + setting termIndexInterval to IndexWriter via SolrIndexConfig. (Jason Rutherglen, hossman, gsingers) + +18. SOLR-846: DIH: Reduce memory consumption during delta import by removing + keys when used (Ricky Leung, Noble Paul via shalin) + +19. SOLR-974: DataImportHandler skips commit if no data has been updated. + (Wojtek Piaseczny, shalin) + +20. SOLR-1004: DIH: Check for abort more frequently during delta-imports. + (Marc Sturlese, shalin) + +21. SOLR-1098: DIH DateFormatTransformer can cache the format objects. + (Noble Paul via shalin) + +22. SOLR-1465: Replaced string concatenations with StringBuilder append + calls in DIH XPathRecordReader. (Mark Miller, shalin) + +Bug Fixes +---------------------- + 1. SOLR-774: Fixed logging level display (Sean Timm via Otis Gospodnetic) + + 2. SOLR-771: CoreAdminHandler STATUS should display 'normalized' paths (koji, hossman, shalin) + + 3. SOLR-532: WordDelimiterFilter now respects payloads and other attributes of the original Token by + using Token.clone() (Tricia Williams, gsingers) + + 4. SOLR-805: DisMax queries are not being cached in QueryResultCache (Todd Feak via koji) + + 5. SOLR-751: WordDelimiterFilter didn't adjust the start offset of single + tokens that started with delimiters, leading to incorrect highlighting. + (Stefan Oestreicher via yonik) + + 7. SOLR-843: SynonymFilterFactory cannot handle multiple synonym files correctly (koji) + + 8. SOLR-840: BinaryResponseWriter does not handle incompatible data in fields (Noble Paul via shalin) + + 9. SOLR-803: CoreAdminRequest.createCore fails because name parameter isn't set (Sean Colombo via ryan) + +10. SOLR-869: Fix file descriptor leak in SolrResourceLoader#getLines (Mark Miller, shalin) + +11. SOLR-872: Better error message for incorrect copyField destination (Noble Paul via shalin) + +12. SOLR-879: Enable position increments in the query parser and fix the + example schema to enable position increments for the stop filter in + both the index and query analyzers to fix the bug with phrase queries + with stopwords. (yonik) + +13. SOLR-836: Add missing "a" to the example stopwords.txt (yonik) + +14. SOLR-892: Fix serialization of booleans for PHPSerializedResponseWriter + (yonik) + +15. SOLR-898: Fix null pointer exception for the JSON response writer + based formats when nl.json=arrarr with null keys. (yonik) + +16. SOLR-901: FastOutputStream ignores write(byte[]) call. (Noble Paul via shalin) + +17. SOLR-807: BinaryResponseWriter writes fieldType.toExternal if it is not a supported type, + otherwise it writes fieldType.toObject. This fixes the bug with encoding/decoding UUIDField. + (koji, Noble Paul, shalin) + +18. SOLR-863: SolrCore.initIndex should close the directory it gets for clearing the lock and + use the DirectoryFactory. (Mark Miller via shalin) + +19. SOLR-802: Fix a potential null pointer error in the distributed FacetComponent + (David Bowen via ryan) + +20. SOLR-346: Use perl regex to improve accuracy of finding latest snapshot in snapinstaller (billa) + +21. SOLR-830: Use perl regex to improve accuracy of finding latest snapshot in snappuller (billa) + +22. SOLR-897: Fixed Argument list too long error when there are lots of snapshots/backups (Dan Rosher via billa) + +23. SOLR-925: Fixed highlighting on fields with multiValued="true" and termOffsets="true" (koji) + +24. SOLR-902: FastInputStream#read(byte b[], int off, int len) gives incorrect results when amount left to read is less + than buffer size (Noble Paul via shalin) + +25. SOLR-978: Old files are not removed from slaves after replication (Jaco, Noble Paul, shalin) + +26. SOLR-883: Implicit properties are not set for Cores created through CoreAdmin (Noble Paul via shalin) + +27. SOLR-991: Better error message when parsing solrconfig.xml fails due to malformed XML. Error message notes the name + of the file being parsed. (Michael Henson via shalin) + +28. SOLR-1008: Fix stats.jsp XML encoding for <stat> item entries with ampersands in their names. (ehatcher) + +29. SOLR-976: deleteByQuery is ignored when deleteById is placed prior to deleteByQuery in a <delete>. + Now both delete by id and delete by query can be specified at the same time as follows. + <delete> + <id>05991</id><id>06000</id> + <query>office:Bridgewater</query><query>office:Osaka</query> + </delete> + (koji) + +30. SOLR-1016: HTTP 503 error changes 500 in SolrCore (koji) + +31. SOLR-1015: Incomplete information in replication admin page and http command response when server + is both master and slave i.e. when server is a repeater (Akshay Ukey via shalin) + +32. SOLR-1018: Slave is unable to replicate when server acts as repeater (as both master and slave) + (Akshay Ukey, Noble Paul via shalin) + +33. SOLR-1031: Fix XSS vulnerability in schema.jsp (Paul Lovvik via ehatcher) + +34. SOLR-1064: registry.jsp incorrectly displaying info for last core initialized + regardless of what the current core is. (hossman) + +35. SOLR-1072: absolute paths used in sharedLib attribute were + incorrectly treated as relative paths. (hossman) + +36. SOLR-1104: Fix some rounding errors in LukeRequestHandler's histogram (hossman) + +37. SOLR-1125: Use query analyzer rather than index analyzer for queryFieldType in QueryElevationComponent + (koji) + +38. SOLR-1126: Replicated files have incorrect timestamp (Jian Han Guo, Jeff Newburn, Noble Paul via shalin) + +39. SOLR-1094: Incorrect value of correctlySpelled attribute in some cases (David Smiley, Mark Miller via shalin) + +40. SOLR-965: Better error message when <pingQuery> is not configured. + (Mark Miller via hossman) + +41. SOLR-1135: Java replication creates Snapshot in the directory where Solr was launched (Jianhan Guo via shalin) + +42. SOLR-1138: Query Elevation Component now gracefully handles missing queries. (gsingers) + +43. SOLR-929: LukeRequestHandler should return "dynamicBase" only if the field is dynamic. + (Peter Wolanin, koji) + +44. SOLR-1141: NullPointerException during snapshoot command in java based replication (Jian Han Guo, shalin) + +45. SOLR-1078: Fixes to WordDelimiterFilter to avoid splitting or dropping + international non-letter characters such as non spacing marks. (yonik) + +46. SOLR-825, SOLR-1221: Enables highlighting for range/wildcard/fuzzy/prefix queries if using hl.usePhraseHighlighter=true + and hl.highlightMultiTerm=true. Also make both options default to true. (Mark Miller, yonik) + +47. SOLR-1174: Fix Logging admin form submit url for multicore. (Jacob Singh via shalin) + +48. SOLR-1182: Fix bug in OrdFieldSource#equals which could cause a bug with OrdFieldSource caching + on OrdFieldSource#hashcode collisions. (Mark Miller) + +49. SOLR-1207: equals method should compare this and other of DocList in DocSetBase (koji) + +50. SOLR-1242: Human readable JVM info from system handler does integer cutoff rounding, even when dealing + with GB. Fixed to round to one decimal place. (Jay Hill, Mark Miller) + +51. SOLR-1243: Admin RequestHandlers should not be cached over HTTP. (Mark Miller) + +52. SOLR-1260: Fix implementations of set operations for DocList subclasses + and fix a bug in HashDocSet construction when offset != 0. These bugs + never manifested in normal Solr use and only potentially affect + custom code. (yonik) + +53. SOLR-1171: Fix LukeRequestHandler so it doesn't rely on SolrQueryParser + and report incorrect stats when field names contain characters + SolrQueryParser considers special. + (hossman) + +54. SOLR-1317: Fix CapitalizationFilterFactory to work when keep parameter is not specified. + (ehatcher) + +55. SOLR-1342: CapitalizationFilterFactory uses incorrect term length calculations. + (Robert Muir via Mark Miller) + +56. SOLR-1359: DoubleMetaphoneFilter didn't index original tokens if there was no + alternative, and could incorrectly skip or reorder tokens. (yonik) + +57. SOLR-1360: Prevent PhoneticFilter from producing duplicate tokens. (yonik) + +58. SOLR-1371: LukeRequestHandler/schema.jsp errored if schema had no + uniqueKey field. The new test for this also (hopefully) adds some + future proofing against similar bugs in the future. As a side + effect QueryElevationComponentTest was refactored, and a bug in + that test was found. (hossman) + +59. SOLR-914: General finalize() improvements. No finalizer delegates + to the respective close/destroy method w/o first checking if it's + already been closed/destroyed; if it hasn't a, SEVERE error is + logged first. (noble, hossman) + +60. SOLR-1362: WordDelimiterFilter had inconsistent behavior when setting + the position increment of tokens following a token consisting of all + delimiters, and could additionally lose big position increments. + (Robert Muir, yonik) + +61. SOLR-1091: Jetty's use of CESU-8 for code points outside the BMP + resulted in invalid output from the serialized PHP writer. (yonik) + +62. SOLR-1103: LukeRequestHandler (and schema.jsp) have been fixed to + include the "1" (ie: 2**0) bucket in the term histogram data. + (hossman) + +63. SOLR-1398: Add offset corrections in PatternTokenizerFactory. + (Anders Melchiorsen, koji) + +64. SOLR-1400: Properly handle zero-length tokens in TrimFilter. This + was not a bug in any released version. (Peter Wolanin, gsingers) + +65. SOLR-1071: spellcheck.extendedResults returns an invalid JSON response + when count > 1. To fix, the extendedResults format was changed. + (Uri Boness, yonik) + +66. SOLR-1381: Fixed improper handling of fields that have only term positions and not term offsets during Highlighting (Thorsten Fischer, gsingers) + +67. SOLR-1427: Fixed registry.jsp issue with MBeans (gsingers) + +68. SOLR-1468: SolrJ's XML response parsing threw an exception for null + names, such as those produced when facet.missing=true (yonik) + +69. SOLR-1471: Fixed issue with calculating missing values for facets in single valued cases in Stats Component. + This is not correctly calculated for the multivalued case. (James Miller, gsingers) + +70. SOLR-1481: Fixed omitHeader parameter for PHP ResponseWriter. (Jun Ohtani via billa) + +71. SOLR-1448: Add weblogic.xml to solr webapp to enable correct operation in + WebLogic. (Ilan Rabinovitch via yonik) + +72. SOLR-1504: empty char mapping can cause ArrayIndexOutOfBoundsException in analysis.jsp and co. + (koji) + +73. SOLR-1394: HTMLStripCharFilter split tokens that contained entities and + often calculated offsets incorrectly for entities. + (Anders Melchiorsen via yonik) + +74. SOLR-1517: Admin pages could stall waiting for localhost name resolution + if reverse DNS wasn't configured; this was changed so the DNS resolution + is attempted only once the first time an admin page is loaded. + (hossman) + +75. SOLR-1529: More than 8 deleteByQuery commands in a single request + caused an error to be returned, although the deletes were + still executed. (asmodean via yonik) + +76. SOLR-800: Deep copy collections to avoid ConcurrentModificationException + in XPathEntityprocessor while streaming + (Kyle Morrison, Noble Paul via shalin) + +77. SOLR-823: Request parameter variables ${dataimporter.request.xxx} are not + resolved in DIH (Mck SembWever, Noble Paul, shalin) + +78. SOLR-728: Add synchronization to avoid race condition of multiple DIH + imports working concurrently (Walter Ferrara, shalin) + +79. SOLR-742: Add ability to create dynamic fields with custom + DataImportHandler transformers (Wojtek Piaseczny, Noble Paul, shalin) + +80. SOLR-832: Rows parameter is not honored in DIH non-debug mode and can + abort a running import in debug mode. (Akshay Ukey, shalin) + +81. SOLR-838: The DIH VariableResolver obtained from a DataSource's context + does not have current data. (Noble Paul via shalin) + +82. SOLR-864: DataImportHandler does not catch and log Errors (shalin) + +83. SOLR-873: Fix case-sensitive field names and columns (Jon Baer, shalin) + +84. SOLR-893: Unable to delete documents via SQL and deletedPkQuery with + deltaimport (Dan Rosher via shalin) + +85. SOLR-888: DIH DateFormatTransformer cannot convert non-string type + (Amit Nithian via shalin) + +86. SOLR-841: DataImportHandler should throw exception if a field does not + have column attribute (Michael Henson, shalin) + +87. SOLR-884: CachedSqlEntityProcessor should check if the cache key is + present in the query results (Noble Paul via shalin) + +88. SOLR-985: Fix thread-safety issue with DIH TemplateString for concurrent + imports with multiple cores. (Ryuuichi Kumai via shalin) + +89. SOLR-999: DIH XPathRecordReader fails on XMLs with nodes mixed with + CDATA content. (Fergus McMenemie, Noble Paul via shalin) + +90. SOLR-1000: DIH FileListEntityProcessor should not apply fileName filter to + directory names. (Fergus McMenemie via shalin) + +91. SOLR-1009: Repeated column names result in duplicate values. + (Fergus McMenemie, Noble Paul via shalin) + +92. SOLR-1017: Fix DIH thread-safety issue with last_index_time for concurrent + imports in multiple cores due to unsafe usage of SimpleDateFormat by + multiple threads. (Ryuuichi Kumai via shalin) + +93. SOLR-1024: Calling abort on DataImportHandler import commits data instead + of calling rollback. (shalin) + +94. SOLR-1037: DIH should not add null values in a row returned by + EntityProcessor to documents. (shalin) + +95. SOLR-1040: DIH XPathEntityProcessor fails with an xpath like + /feed/entry/link[@type='text/html']/@href (Noble Paul via shalin) + +96. SOLR-1042: Fix memory leak in DIH by making TemplateString non-static + member in VariableResolverImpl (Ryuuichi Kumai via shalin) + +97. SOLR-1053: IndexOutOfBoundsException in DIH SolrWriter.getResourceAsString + when size of data-config.xml is a multiple of 1024 bytes. + (Herb Jiang via shalin) + +98. SOLR-1077: IndexOutOfBoundsException with useSolrAddSchema in DIH + XPathEntityProcessor. (Sam Keen, Noble Paul via shalin) + +99. SOLR-1080: DIH RegexTransformer should not replace if regex is not matched. + (Noble Paul, Fergus McMenemie via shalin) + +100.SOLR-1090: DataImportHandler should load the data-config.xml using UTF-8 + encoding. (Rui Pereira, shalin) + +101.SOLR-1146: ConcurrentModificationException in DataImporter.getStatusMessages + (Walter Ferrara, Noble Paul via shalin) + +102.SOLR-1229: Fixes for DIH deletedPkQuery, particularly when using + transformed Solr unique id's + (Lance Norskog, Noble Paul via ehatcher) + +103.SOLR-1286: Fix the IH commit parameter always defaulting to "true" even + if "false" is explicitly passed in. (Jay Hill, Noble Paul via ehatcher) + +104.SOLR-1323: Reset XPathEntityProcessor's $hasMore/$nextUrl when fetching + next URL (noble, ehatcher) + +105.SOLR-1450: DIH: Jdbc connection properties such as batchSize are not + applied if the driver jar is placed in solr_home/lib. + (Steve Sun via shalin) + +106.SOLR-1474: DIH Delta-import should run even if last_index_time is not set. + (shalin) + + +Other Changes +---------------------- + 1. Upgraded to Lucene 2.4.0 (yonik) + + 2. SOLR-805: Upgraded to Lucene 2.9-dev (r707499) (koji) + + 3. DumpRequestHandler (/debug/dump): changed 'fieldName' to 'sourceInfo'. (ehatcher) + + 4. SOLR-852: Refactored common code in CSVRequestHandler and XMLUpdateRequestHandler (gsingers, ehatcher) + + 5. SOLR-871: Removed dependency on stax-utils.jar. If you using solr.jar and running + java 6, you can also remove woodstox and geronimo. (ryan) + + 6. SOLR-465: Upgraded to Lucene 2.9-dev (r719351) (shalin) + + 7. SOLR-889: Upgraded to commons-io-1.4.jar and commons-fileupload-1.2.1.jar (ryan) + + 8. SOLR-875: Upgraded to Lucene 2.9-dev (r723985) and consolidated the BitSet implementations (Michael Busch, gsingers) + + 9. SOLR-819: Upgraded to Lucene 2.9-dev (r724059) to get access to Arabic public constructors (gsingers) + +10. SOLR-900: Moved solrj into /src/solrj. The contents of solr-common.jar is now included + in the solr-solrj.jar. (ryan) + +11. SOLR-924: Code cleanup: make all existing finalize() methods call + super.finalize() in a finally block. All current instances extend + Object, so this doesn't fix any bugs, but helps protect against + future changes. (Kay Kay via hossman) + +12. SOLR-885: NamedListCodec is renamed to JavaBinCodec and returns Object instead of NamedList. + (Noble Paul, yonik via shalin) + +13. SOLR-84: Use new Solr logo in admin (Michiel via koji) + +14. SOLR-981: groupId for Woodstox dependency in maven solrj changed to org.codehaus.woodstox (Tim Taranov via shalin) + +15. Upgraded to Lucene 2.9-dev r738218 (yonik) + +16. SOLR-959: Refactored TestReplicationHandler to remove hardcoded port numbers (hossman, Akshay Ukey via shalin) + +17. Upgraded to Lucene 2.9-dev r742220 (yonik) + +18. SOLR-1022: Better "ignored" field in example schema.xml (Peter Wolanin via hossman) + +19. SOLR-967: New type-safe constructor for NamedList (Kay Kay via hossman) + +20. SOLR-1036: Change default QParser from "lucenePlusSort" to "lucene" to + reduce confusion of semicolon splitting behavior when no sort param is + specified (hossman) + +21. Upgraded to Lucene 2.9-dev r752164 (shalin) + +22. SOLR-1068: Use fsync on replicated index and configuration files (yonik, Noble Paul, shalin) + +23. SOLR-952: Cleanup duplicated code in deprecated HighlightingUtils (hossman) + +24. Upgraded to Lucene 2.9-dev r764281 (shalin) + +25. SOLR-1079: Rename omitTf to omitTermFreqAndPositions (shalin) + +26. SOLR-804: Added Lucene's misc contrib JAR (rev 764281). (gsingers) + +27. Upgraded to Lucene 2.9-dev r768228 (shalin) + +28. Upgraded to Lucene 2.9-dev r768336 (shalin) + +29. SOLR-997: Wait for a longer time for slave to complete replication in TestReplicationHandler + (Mark Miller via shalin) + +30. SOLR-748: FacetComponent helper classes are made public as an experimental API. + (Wojtek Piaseczny via shalin) + +31. Upgraded to Lucene 2.9-dev 773862 (Mark Miller) + +32. Upgraded to Lucene 2.9-dev r776177 (shalin) + +33. SOLR-1149: Made QParserPlugin and related classes extendible as an experimental API. + (Kaktu Chakarabati via shalin) + +34. Upgraded to Lucene 2.9-dev r779312 (yonik) + +35. SOLR-786: Refactor DisMaxQParser to allow overriding certain features of DisMaxQParser + (Wojciech Biela via shalin) + +36. SOLR-458: Add equals and hashCode methods to NamedList (Stefan Rinner, shalin) + +37. SOLR-1184: Add option in solrconfig to open a new IndexReader rather than + using reopen. Done mainly as a fail-safe in the case that a user runs into + a reopen bug/issue. (Mark Miller) + +38. SOLR-1215 use double quotes to enclose attributes in solr.xml (noble) + +39. SOLR-1151: add dynamic copy field and maxChars example to example schema.xml. + (Peter Wolanin, Mark Miller) + +40. SOLR-1233: remove /select?qt=/whatever restriction on /-prefixed request handlers. + (ehatcher) + +41. SOLR-1257: logging.jsp has been removed and now passes through to the + hierarchical log level tool added in Solr 1.3. Users still + hitting "/admin/logging.jsp" should switch to "/admin/logging". + (hossman) + +42. Upgraded to Lucene 2.9-dev r794238. Other changes include: + - LUCENE-1614 - Use Lucene's DocIdSetIterator.NO_MORE_DOCS as the sentinel value. + - LUCENE-1630 - Add acceptsDocsOutOfOrder method to Collector implementations. + - LUCENE-1673, LUCENE-1701 - Trie has moved to Lucene core and renamed to NumericRangeQuery. + - LUCENE-1662, LUCENE-1687 - Replace usage of ExtendedFieldCache by FieldCache. + (shalin) + +42. SOLR-1241: Solr's CharFilter has been moved to Lucene. Remove CharFilter and related classes + from Solr and use Lucene's corresponding code (koji via shalin) + +43. SOLR-1261: Lucene trunk renamed RangeQuery & Co to TermRangeQuery (Uwe Schindler via shalin) + +44. Upgraded to Lucene 2.9-dev r801856 (Mark Miller) + +45. SOLR-1276: Added StatsComponentTest (Rafał Kuć, gsingers) + +46. SOLR-1377: The TokenizerFactory API has changed to explicitly return a Tokenizer + rather then a TokenStream (that may be or may not be a Tokenizer). This change + is required to take advantage of the Token reuse improvements in lucene 2.9. (ryan) + +47. SOLR-1410: Log a warning if the deprecated charset option is used + on GreekLowerCaseFilterFactory, RussianStemFilterFactory, + RussianLowerCaseFilterFactory or RussianLetterTokenizerFactory. + (Robert Muir via hossman) + +48. SOLR-1423: Due to LUCENE-1906, Solr's tokenizer should use Tokenizer.correctOffset() instead of CharStream.correctOffset(). + (Uwe Schindler via koji) + +49. SOLR-1319, SOLR-1345: Upgrade Solr Highlighter classes to new Lucene Highlighter API. This upgrade has + resulted in a back compat break in the DefaultSolrHighlighter class - getQueryScorer is no longer + protected. If you happened to be overriding that method in custom code, overide getHighlighter instead. + Also, HighlightingUtils#getQueryScorer has been removed as it was deprecated and backcompat has been + broken with it anyway. (Mark Miller) + +50. SOLR-1357 SolrInputDocument cannot process dynamic fields (Lars Grote via noble) + +51. SOLR-1075: Upgrade to Tika 0.3. See http://www.apache.org/dist/lucene/tika/CHANGES-0.3.txt (gsingers) + +52. SOLR-1310: Upgrade to Tika 0.4. Note there are some differences in + detecting Languages now in extracting request handler. + See http://www.lucidimagination.com/search/document/d6f1899a85b2a45c/vote_apache_tika_0_4_release_candidate_2#d6f1899a85b2a45c + for discussion on language detection. + See http://www.apache.org/dist/lucene/tika/CHANGES-0.4.txt. (gsingers) + +53. SOLR-782: DIH: Refactored SolrWriter to make it a concrete class and + removed wrappers over SolrInputDocument. Refactored to load Evaluators + lazily. Removed multiple document nodes in the configuration xml. Removed + support for 'default' variables, they are automatically available as + request parameters. (Noble Paul via shalin) + +54. SOLR-964: DIH: XPathEntityProcessor now ignores DTD validations + (Fergus McMenemie, Noble Paul via shalin) + +55. SOLR-1029: DIH: Standardize Evaluator parameter parsing and added helper + functions for parsing all evaluator parameters in a standard way. + (Noble Paul, shalin) + +56. SOLR-1081: Change DIH EventListener to be an interface so that components + such as an EntityProcessor or a Transformer can act as an event listener. + (Noble Paul, shalin) + +57. SOLR-1027: DIH: Alias the 'dataimporter' namespace to a shorter name 'dih'. + (Noble Paul via shalin) + +58. SOLR-1084: Better error reporting when DIH entity name is a reserved word + and data-config.xml root node is not <dataConfig>. + (Noble Paul via shalin) + +59. SOLR-1087: Deprecate 'where' attribute in CachedSqlEntityProcessor in + favor of cacheKey and cacheLookup. (Noble Paul via shalin) + +60. SOLR-969: Change the FULL_DUMP, DELTA_DUMP, FIND_DELTA constants in DIH + Context to String. Change Context.currentProcess() to return a string + instead of an integer. (Kay Kay, Noble Paul, shalin) + +61. SOLR-1120: Simplified DIH EntityProcessor API by moving logic for applying + transformers and handling multi-row outputs from Transformers into an + EntityProcessorWrapper class. The behavior of the method + EntityProcessor#destroy has been modified to be called once per parent-row + at the end of row. A new method EntityProcessor#close is added which is + called at the end of import. A new method + Context#getResolvedEntityAttribute is added which returns the resolved + value of an entity's attribute. Introduced a DocWrapper which takes care + of maintaining document level session variables. + (Noble Paul, shalin) + +62. SOLR-1265: Add DIH variable resolving for URLDataSource properties like + baseUrl. (Chris Eldredge via ehatcher) + +63. SOLR-1269: Better error messages from DIH JdbcDataSource when JDBC Driver + name or SQL is incorrect. (ehatcher, shalin) + + +Build +---------------------- + 1. SOLR-776: Added in ability to sign artifacts via Ant for releases (gsingers) + + 2. SOLR-854: Added run-example target (Mark Miller via ehatcher) + + 3. SOLR-1054:Fix dist-src target for DataImportHandler (Ryuuichi Kumai via shalin) + + 4. SOLR-1219: Added proxy.setup target (koji) + + 5. SOLR-1386: In build.xml, use longfile="gnu" in tar task to avoid warnings about long file names + (Mark Miller via shalin) + + 6. SOLR-1441: Make it possible to run all tests in a package (shalin) + + +Documentation +---------------------- + 1. SOLR-789: The javadoc of RandomSortField is not readable (Nicolas Lalevée via koji) + + 2. SOLR-962: Note about null handling in ModifiableSolrParams.add javadoc + (Kay Kay via hossman) + + 3. SOLR-1409: Added Solr Powered By Logos + + 4. SOLR-1369: Add HSQLDB Jar to example-DIH, unzip database and update + instructions. + + +================== Release 1.3.0 ================== + +Upgrading from Solr 1.2 +----------------------- +IMPORTANT UPGRADE NOTE: In a master/slave configuration, all searchers/slaves +should be upgraded before the master! If the master were to be updated +first, the older searchers would not be able to read the new index format. + +The Porter snowball based stemmers in Lucene were updated (LUCENE-1142), +and are not guaranteed to be backward compatible at the index level +(the stem of certain words may have changed). Re-indexing is recommended. + +Older Apache Solr installations can be upgraded by replacing +the relevant war file with the new version. No changes to configuration +files should be needed. + +This version of Solr contains a new version of Lucene implementing +an updated index format. This version of Solr/Lucene can still read +and update indexes in the older formats, and will convert them to the new +format on the first index change. Be sure to backup your index before +upgrading in case you need to downgrade. + +Solr now recognizes HTTP Request headers related to HTTP Caching (see +RFC 2616 sec13) and will by default respond with "304 Not Modified" +when appropriate. This should only affect users who access Solr via +an HTTP Cache, or via a Web-browser that has an internal cache, but if +you wish to suppress this behavior an '<httpCaching never304="true"/>' +option can be added to your solrconfig.xml. See the wiki (or the +example solrconfig.xml) for more details... + http://wiki.apache.org/solr/SolrConfigXml#HTTPCaching + +In Solr 1.2, DateField did not enforce the canonical representation of +the ISO 8601 format when parsing incoming data, and did not generation +the canonical format when generating dates from "Date Math" strings +(particularly as it pertains to milliseconds ending in trailing zeros). +As a result equivalent dates could not always be compared properly. +This problem is corrected in Solr 1.3, but DateField users that might +have been affected by indexing inconsistent formats of equivalent +dates (ie: 1995-12-31T23:59:59Z vs 1995-12-31T23:59:59.000Z) may want +to consider reindexing to correct these inconsistencies. Users who +depend on some of the the "broken" behavior of DateField in Solr 1.2 +(specificly: accepting any input that ends in a 'Z') should consider +using the LegacyDateField class as a possible alternative. Users that +desire 100% backwards compatibility should consider using the Solr 1.2 +version of DateField. + +Due to some changes in the lifecycle of TokenFilterFactories, users of +Solr 1.2 who have written Java code which constructs new instances of +StopFilterFactory, SynonymFilterFactory, or EnglishProterFilterFactory +will need to modify their code by adding a line like the following +prior to using the factory object... + factory.inform(SolrCore.getSolrCore().getSolrConfig().getResourceLoader()); +These lifecycle changes do not affect people who use Solr "out of the +box" or who have developed their own TokenFilterFactory plugins. More +info can be found in SOLR-594. + +The python client that used to ship with Solr is no longer included in +the distribution (see client/python/README.txt). + +Detailed Change List +-------------------- + +New Features + 1. SOLR-69: Adding MoreLikeThisHandler to search for similar documents using + lucene contrib/queries MoreLikeThis. MoreLikeThis is also available from + the StandardRequestHandler using ?mlt=true. (bdelacretaz, ryan) + + 2. SOLR-253: Adding KeepWordFilter and KeepWordFilterFactory. A TokenFilter + that keeps tokens with text in the registered keeplist. This behaves like + the inverse of StopFilter. (ryan) + + 3. SOLR-257: WordDelimiterFilter has a new parameter splitOnCaseChange, + which can be set to 0 to disable splitting "PowerShot" => "Power" "Shot". + (klaas) + + 4. SOLR-193: Adding SolrDocument and SolrInputDocument to represent documents + outside of the lucene Document infrastructure. This class will be used + by clients and for processing documents. (ryan) + + 5. SOLR-244: Added ModifiableSolrParams - a SolrParams implementation that + help you change values after initialization. (ryan) + + 6. SOLR-20: Added a java client interface with two implementations. One + implementation uses commons httpclient to connect to solr via HTTP. The + other connects to solr directly. Check client/java/solrj. This addition + also includes tests that start jetty and test a connection using the full + HTTP request cycle. (Darren Erik Vengroff, Will Johnson, ryan) + + 7. SOLR-133: Added StaxUpdateRequestHandler that uses StAX for XML parsing. + This implementation has much better error checking and lets you configure + a custom UpdateRequestProcessor that can selectively process update + requests depending on the request attributes. This class will likely + replace XmlUpdateRequestHandler. (Thorsten Scherler, ryan) + + 8. SOLR-264: Added RandomSortField, a utility field with a random sort order. + The seed is based on a hash of the field name, so a dynamic field + of this type is useful for generating different random sequences. + This field type should only be used for sorting or as a value source + in a FunctionQuery (ryan, hossman, yonik) + + 9. SOLR-266: Adding show=schema to LukeRequestHandler to show the parsed + schema fields and field types. (ryan) + +10. SOLR-133: The UpdateRequestHandler now accepts multiple delete options + within a single request. For example, sending: + <delete><id>1</id><id>2</id></delete> will delete both 1 and 2. (ryan) + +11. SOLR-269: Added UpdateRequestProcessor plugin framework. This provides + a reasonable place to process documents after they are parsed and + before they are committed to the index. This is a good place for custom + document manipulation or document based authorization. (yonik, ryan) + +12. SOLR-260: Converting to a standard PluginLoader framework. This reworks + RequestHandlers, FieldTypes, and QueryResponseWriters to share the same + base code for loading and initializing plugins. This adds a new + configuration option to define the default RequestHandler and + QueryResponseWriter in XML using default="true". (ryan) + +13. SOLR-225: Enable pluggable highlighting classes. Allow configurable + highlighting formatters and Fragmenters. (ryan) + +14. SOLR-273/376/452/516: Added hl.maxAnalyzedChars highlighting parameter, defaulting + to 50k, hl.alternateField, which allows the specification of a backup + field to use as summary if no keywords are matched, and hl.mergeContiguous, + which combines fragments if they are adjacent in the source document. + (klaas, Grant Ingersoll, Koji Sekiguchi via klaas) + +15. SOLR-291: Control maximum number of documents to cache for any entry + in the queryResultCache via queryResultMaxDocsCached solrconfig.xml + entry. (Koji Sekiguchi via yonik) + +16. SOLR-240: New <lockType> configuration setting in <mainIndex> and + <indexDefaults> blocks supports all Lucene builtin LockFactories. + 'single' is recommended setting, but 'simple' is default for total + backwards compatibility. + (Will Johnson via hossman) + +17. SOLR-248: Added CapitalizationFilterFactory that creates tokens with + normalized capitalization. This filter is useful for facet display, + but will not work with a prefix query. (ryan) + SOLR-468: Change to the semantics to keep the original token, not the + token in the Map. Also switched to use Lucene's new reusable token + capabilities. (gsingers) + +18. SOLR-307: Added NGramFilterFactory and EdgeNGramFilterFactory. + (Thomas Peuss via Otis Gospodnetic) + +19. SOLR-305: analysis.jsp can be given a fieldtype instead of a field + name. (hossman) + +20. SOLR-102: Added RegexFragmenter, which splits text for highlighting + based on a given pattern. (klaas) + +21. SOLR-258: Date Faceting added to SimpleFacets. Facet counts + computed for ranges of size facet.date.gap (a DateMath expression) + between facet.date.start and facet.date.end. (hossman) + +22. SOLR-196: A PHP serialized "phps" response writer that returns a + serialized array that can be used with the PHP function unserialize, + and a PHP response writer "php" that may be used by eval. + (Nick Jenkin, Paul Borgermans, Pieter Berkel via yonik) + +23. SOLR-308: A new UUIDField class which accepts UUID string values, + as well as the special value of "NEW" which triggers generation of + a new random UUID. + (Thomas Peuss via hossman) + +24. SOLR-349: New FunctionQuery functions: sum, product, div, pow, log, + sqrt, abs, scale, map. Constants may now be used as a value source. + (yonik) + +25. SOLR-359: Add field type className to Luke response, and enabled access + to the detailed field information from the solrj client API. + (Grant Ingersoll via ehatcher) + +26. SOLR-334: Pluggable query parsers. Allows specification of query + type and arguments as a prefix on a query string. (yonik) + +27. SOLR-351: External Value Source. An external file may be used + to specify the values of a field, currently usable as + a ValueSource in a FunctionQuery. (yonik) + +28. SOLR-395: Many new features for the spell checker implementation, including + an extended response mode with much richer output, multi-word spell checking, + and a bevy of new and renamed options (see the wiki). + (Mike Krimerman, Scott Taber via klaas). + +29. SOLR-408: Added PingRequestHandler and deprecated SolrCore.getPingQueryRequest(). + Ping requests should be configured using standard RequestHandler syntax in + solrconfig.xml rather then using the <pingQuery></pingQuery> syntax. + (Karsten Sperling via ryan) + +30. SOLR-281: Added a 'Search Component' interface and converted StandardRequestHandler + and DisMaxRequestHandler to use this framework. + (Sharad Agarwal, Henri Biestro, yonik, ryan) + +31. SOLR-176: Add detailed timing data to query response output. The SearchHandler + interface now returns how long each section takes. (klaas) + +32. SOLR-414: Plugin initialization now supports SolrCore and ResourceLoader "Aware" + plugins. Plugins that implement SolrCoreAware or ResourceLoaderAware are + informed about the SolrCore/ResourceLoader. (Henri Biestro, ryan) + +33. SOLR-350: Support multiple SolrCores running in the same solr instance and allows + runtime runtime management for any running SolrCore. If a solr.xml file exists + in solr.home, this file is used to instanciate multiple cores and enables runtime + core manipulation. For more informaion see: http://wiki.apache.org/solr/CoreAdmin + (Henri Biestro, ryan) + +34. SOLR-447: Added an single request handler that will automatically register all + standard admin request handlers. This replaces the need to register (and maintain) + the set of admin request handlers. Assuming solrconfig.xml includes: + <requestHandler name="/admin/" class="org.apache.solr.handler.admin.AdminHandlers" /> + This will register: Luke/SystemInfo/PluginInfo/ThreadDump/PropertiesRequestHandler. + (ryan) + +35. SOLR-142: Added RawResponseWriter and ShowFileRequestHandler. This returns config + files directly. If AdminHandlers are configured, this will be added automatically. + The jsp files /admin/get-file.jsp and /admin/raw-schema.jsp have been deprecated. + The deprecated <admin><gettableFiles> will be automatically registered with + a ShowFileRequestHandler instance for backwards compatibility. (ryan) + +36. SOLR-446: TextResponseWriter can write SolrDocuments and SolrDocumentLists the + same way it writes Document and DocList. (yonik, ryan) + +37. SOLR-418: Adding a query elevation component. This is an optional component to + elevate some documents to the top positions (or exclude them) for a given query. + (ryan) + +38. SOLR-478: Added ability to get back unique key information from the LukeRequestHandler. + (gsingers) + +39. SOLR-127: HTTP Caching awareness. Solr now recognizes HTTP Request + headers related to HTTP Caching (see RFC 2616 sec13) and will respond + with "304 Not Modified" when appropriate. New options have been added + to solrconfig.xml to influence this behavior. + (Thomas Peuss via hossman) + +40. SOLR-303: Distributed Search over HTTP. Specification of shards + argument causes Solr to query those shards and merge the results + into a single response. Querying, field faceting (sorted only), + query faceting, highlighting, and debug information are supported + in distributed mode. + (Sharad Agarwal, Patrick O'Leary, Sabyasachi Dalal, Stu Hood, + Jayson Minard, Lars Kotthoff, ryan, yonik) + +41. SOLR-356: Pluggable functions (value sources) that allow + registration of new functions via solrconfig.xml + (Doug Daniels via yonik) + +42. SOLR-494: Added cool admin Ajaxed schema explorer. + (Greg Ludington via ehatcher) + +43. SOLR-497: Added date faceting to the QueryResponse in SolrJ + and QueryResponseTest (Shalin Shekhar Mangar via gsingers) + +44. SOLR-486: Binary response format, faster and smaller + than XML and JSON response formats (use wt=javabin). + BinaryResponseParser for utilizing the binary format via SolrJ + and is now the default. + (Noble Paul, yonik) + +45. SOLR-521: StopFilterFactory support for "enablePositionIncrements" + (Walter Ferrara via hossman) + +46. SOLR-557: Added SolrCore.getSearchComponents() to return an unmodifiable Map. (gsingers) + +47. SOLR-516: Added hl.maxAlternateFieldLength parameter, to set max length for hl.alternateField + (Koji Sekiguchi via klaas) + +48. SOLR-319: Changed SynonymFilterFactory to "tokenize" synonyms file. + To use a tokenizer, specify "tokenizerFactory" attribute in <filter>. + For example: + <tokenizer class="solr.CJKTokenizerFactory"/> + <filter class="solr.SynonymFilterFactory" synonyms="synonyms.txt" expand="true" + ignoreCase="true" tokenizerFactory="solr.CJKTokenizerFactory"/> + (koji) + +49. SOLR-515: Added SimilarityFactory capability to schema.xml, + making config file parameters usable in the construction of + the global Lucene Similarity implementation. + (ehatcher) + +50. SOLR-536: Add a DocumentObjectBinder to solrj that converts Objects to and + from SolrDocuments. (Noble Paul via ryan) + +51. SOLR-595: Add support for Field level boosting in the MoreLikeThis Handler. + (Tom Morton, gsingers) + +52. SOLR-572: Added SpellCheckComponent and org.apache.solr.spelling package to support more spell + checking functionality. Also includes ability to add your own SolrSpellChecker implementation that + plugs in. See http://wiki.apache.org/solr/SpellCheckComponent for more details + (Shalin Shekhar Mangar, Bojan Smid, gsingers) + +53. SOLR-679: Added accessor methods to Lucene based spell checkers (gsingers) + +54. SOLR-423: Added Request Handler close hook notification so that RequestHandlers can be notified + when a core is closing. (gsingers, ryan) + +55. SOLR-603: Added ability to partially optimize. (gsingers) + +56. SOLR-483: Add byte/short sorting support (gsingers) + +57. SOLR-14: Add preserveOriginal flag to WordDelimiterFilter + (Geoffrey Young, Trey Hyde, Ankur Madnani, yonik) + +58. SOLR-502: Add search timeout support. (Sean Timm via yonik) + +59. SOLR-605: Add the ability to register callbacks programatically (ryan, Noble Paul) + +60. SOLR-610: hl.maxAnalyzedChars can be -1 to highlight everything (Lars Kotthoff via klaas) + +61. SOLR-522: Make analysis.jsp show payloads. (Tricia Williams via yonik) + +62. SOLR-611: Expose sort_values returned by QueryComponent in SolrJ's QueryResponse + (Dan Rosher via shalin) + +63. SOLR-256: Support exposing Solr statistics through JMX (Sharad Agrawal, shalin) + +64. SOLR-666: Expose warmup time in statistics for SolrIndexSearcher and LRUCache (shalin) + +65. SOLR-663: Allow multiple files for stopwords, keepwords, protwords and synonyms + (Otis Gospodnetic, shalin) + +66. SOLR-469: Added DataImportHandler as a contrib project which makes indexing data from Databases, + XML files and HTTP data sources into Solr quick and easy. Includes API and implementations for + supporting multiple data sources, processors and transformers for importing data. Supports full + data imports as well as incremental (delta) indexing. See http://wiki.apache.org/solr/DataImportHandler + for more details. (Noble Paul, shalin) + +67. SOLR-622: SpellCheckComponent supports auto-loading indices on startup and optionally, (re)builds + indices on newSearcher event, if configured in solrconfig.xml (shalin) + +68. SOLR-554: Hierarchical JDK log level selector for SOLR Admin replaces logging.jsp + (Sean Timm via shalin) + +69. SOLR-506: Emitting HTTP Cache headers can be enabled or disabled through configuration on a + per-handler basis (shalin) + +70. SOLR-716: Added support for properties in configuration files. Properties can be specified in + solr.xml and can be used in solrconfig.xml and schema.xml (Henri Biestro, hossman, ryan, shalin) + +71. SOLR-1129 : Support binding dynamic fields to beans in SolrJ (Avlesh Singh , noble) + +72. SOLR-920 : Cache and reuse IndexSchema . A new attribute added in solr.xml called 'shareSchema' (noble) + +73. SOLR-700: DIH: Allow configurable locales through a locale attribute in + fields for NumberFormatTransformer. (Stefan Oestreicher, shalin) + +Changes in runtime behavior + 1. SOLR-559: use Lucene updateDocument, deleteDocuments methods. This + removes the maxBufferedDeletes parameter added by SOLR-310 as Lucene + now manages the deletes. This provides slightly better indexing + performance and makes overwrites atomic, eliminating the possibility of + a crash causing duplicates. (yonik) + + 2. SOLR-689 / SOLR-695: If you have used "MultiCore" functionality in an unreleased + version of 1.3-dev, many classes and configs have been renamed for the official + 1.3 release. Speciffically, solr.xml has replaced multicore.xml, and uses a slightly + different syntax. The solrj classes: MultiCore{Request/Response/Params} have been + renamed: CoreAdmin{Request/Response/Params} (hossman, ryan, Henri Biestro) + + 3. SOLR-647: reference count the SolrCore uses to prevent a premature + close while a core is still in use. (Henri Biestro, Noble Paul, yonik) + + 4. SOLR-737: SolrQueryParser now uses a ConstantScoreQuery for wildcard + queries that prevent an exception from being thrown when the number + of matching terms exceeds the BooleanQuery clause limit. (yonik) + +Optimizations + 1. SOLR-276: improve JSON writer speed. (yonik) + + 2. SOLR-310: bound and reduce memory usage by providing <maxBufferedDeletes> parameter, + which flushes deleted without forcing the user to use <commit/> for this purpose. + (klaas) + + 3. SOLR-348: short-circuit faceting if less than mincount docs match. (yonik) + + 4. SOLR-354: Optimize removing all documents. Now when a delete by query + of *:* is issued, the current index is removed. (yonik) + + 5. SOLR-377: Speed up response writers. (yonik) + + 6. SOLR-342: Added support into the SolrIndexWriter for using several new features of the new + LuceneIndexWriter, including: setRAMBufferSizeMB(), setMergePolicy(), setMergeScheduler. + Also, added support to specify Lucene's autoCommit functionality (not to be confused with Solr's + similarily named autoCommit functionality) via the <luceneAutoCommit> config. item. See the test + and example solrconfig.xml <indexDefaults> section for usage. Performance during indexing should + be significantly increased by moving up to 2.3 due to Lucene's new indexing capabilities. + Furthermore, the setRAMBufferSizeMB makes it more logical to decide on tuning factors related to + indexing. For best performance, leave the mergePolicy and mergeScheduler as the defaults and set + ramBufferSizeMB instead of maxBufferedDocs. The best value for this depends on the types of + documents in use. 32 should be a good starting point, but reports have shown up to 48 MB provides + good results. Note, it is acceptable to set both ramBufferSizeMB and maxBufferedDocs, and Lucene + will flush based on whichever limit is reached first. (gsingers) + + 7. SOLR-330: Converted TokenStreams to use Lucene's new char array based + capabilities. (gsingers) + + 8. SOLR-624: Only take snapshots if there are differences to the index (Richard Trey Hyde via gsingers) + + 9. SOLR-587: Delete by Query performance greatly improved by using + new underlying Lucene IndexWriter implementation. (yonik) + +10. SOLR-730: Use read-only IndexReaders that don't synchronize + isDeleted(). This will speed up function queries and *:* queries + as well as improve their scalability on multi-CPU systems. + (Mark Miller via yonik) + +Bug Fixes + 1. Make TextField respect sortMissingFirst and sortMissingLast fields. + (J.J. Larrea via yonik) + + 2. autoCommit/maxDocs was not working properly when large autoCommit/maxTime + was specified (klaas) + + 3. SOLR-283: autoCommit was not working after delete. (ryan) + + 4. SOLR-286: ContentStreamBase was not using default encoding for getBytes() + (Toru Matsuzawa via ryan) + + 5. SOLR-292: Fix MoreLikeThis facet counting. (Pieter Berkel via ryan) + + 6. SOLR-297: Fix bug in RequiredSolrParams where requiring a field + specific param would fail if a general default value had been supplied. + (hossman) + + 7. SOLR-331: Fix WordDelimiterFilter handling of offsets for synonyms or + other injected tokens that can break highlighting. (yonik) + + 8. SOLR-282: Snapshooter does not work on Solaris and OS X since the cp command + there does not have the -l option. Also updated commit/optimize related + scripts to handle both old and new response format. (bill) + + 9. SOLR-294: Logging of elapsed time broken on Solaris because the date command + there does not support the %s output format. (bill) + +10. SOLR-136: Snappuller - "date -d" and locales don't mix. (Jürgen Hermann via bill) + +11. SOLR-333: Changed distributiondump.jsp to use Solr HOME instead of CWD to set path. + +12. SOLR-393: Removed duplicate contentType from raw-schema.jsp. (bill) + +13. SOLR-413: Requesting a large numbers of documents to be returned (limit) + can result in an out-of-memory exception, even for a small index. (yonik) + +14. The CSV loader incorrectly threw an exception when given + header=true (the default). (ryan, yonik) + +15. SOLR-449: the python and ruby response writers are now able to correctly + output NaN and Infinity in their respective languages. (klaas) + +16. SOLR-42: HTMLStripReader tokenizers now preserve correct source + offsets for highlighting. (Grant Ingersoll via yonik) + +17. SOLR-481: Handle UnknownHostException in _info.jsp (gsingers) + +18. SOLR-324: Add proper support for Long and Doubles in sorting, etc. (gsingers) + +19. SOLR-496: Cache-Control max-age changed to Long so Expires + calculation won't cause overflow. (Thomas Peuss via hossman) + +20. SOLR-535: Fixed typo (Tokenzied -> Tokenized) in schema.jsp (Thomas Peuss via billa) + +21. SOLR-529: Better error messages from SolrQueryParser when field isn't + specified and there is no defaultSearchField in schema.xml + (Lars Kotthoff via hossman) + +22. SOLR-530: Better error messages/warnings when parsing schema.xml: + field using bogus fieldtype and multiple copyFields to a non-multiValue + field. (Shalin Shekhar Mangar via hossman) + +23. SOLR-528: Better error message when defaultSearchField is bogus or not + indexed. (Lars Kotthoff via hossman) + +24. SOLR-533: Fixed tests so they don't use hardcoded port numbers. + (hossman) + +25. SOLR-400: SolrExceptionTest should now handle using OpenDNS as a DNS provider (gsingers) + +26. SOLR-541: Legacy XML update support (provided by SolrUpdateServlet + when no RequestHandler is mapped to "/update") now logs error correctly. + (hossman) + +27. SOLR-267: Changed logging to report number of hits, and also provide a mechanism to add log + messages to be output by the SolrCore via a NamedList toLog member variable. + (Will Johnson, yseeley, gsingers) + + - SOLR-267: Removed adding values to the HTTP headers in SolrDispatchFilter (gsingers) + +28. SOLR-509: Moved firstSearcher event notification to the end of the SolrCore constructor + (Koji Sekiguchi via gsingers) + +29. SOLR-470, SOLR-552, SOLR-544, SOLR-701: Multiple fixes to DateField + regarding lenient parsing of optional milliseconds, and correct + formating using the canonical representation. LegacyDateField has + been added for people who have come to depend on the existing + broken behavior. (hossman, Stefan Oestreicher) + +30. SOLR-539: Fix for non-atomic long counters and a cast fix to avoid divide + by zero. (Sean Timm via Otis Gospodnetic) + +31. SOLR-514: Added explicit media-type with UTF* charset to *.xsl files that + don't already have one. (hossman) + +32. SOLR-505: Give RequestHandlers the possiblity to suppress the generation + of HTTP caching headers. (Thomas Peuss via Otis Gospodnetic) + +33. SOLR-553: Handle highlighting of phrase terms better when + hl.usePhraseHighligher=true URL param is used. + (Bojan Smid via Otis Gospodnetic) + +34. SOLR-590: Limitation in pgrep on Linux platform breaks script-utils fixUser. + (Hannes Schmidt via billa) + +35. SOLR-597: SolrServlet no longer "caches" SolrCore. This was causing + problems in Resin, and could potentially cause problems for customized + usages of SolrServlet. + +36. SOLR-585: Now sets the QParser on the ResponseBuilder (gsingers) + +37. SOLR-604: If the spellchecking path is relative, make it relative to the Solr Data Directory. + (Shalin Shekhar Mangar via gsingers) + +38. SOLR-584: Make stats.jsp and stats.xsl more robust. + (Yousef Ourabi and hossman) + +39. SOLR-443: SolrJ: Declare UTF-8 charset on POSTed parameters + to avoid problems with servlet containers that default to latin-1 + and allow switching of the exact POST mechanism for parameters + via useMultiPartPost in CommonsHttpSolrServer. + (Lars Kotthoff, Andrew Schurman, ryan, yonik) + +40. SOLR-556: multi-valued fields always highlighted in disparate snippets + (Lars Kotthoff via klaas) + +41. SOLR-501: Fix admin/analysis.jsp UTF-8 input for some other servlet + containers such as Tomcat. (Hiroaki Kawai, Lars Kotthoff via yonik) + +42. SOLR-616: SpellChecker accuracy configuration is not applied for FileBasedSpellChecker. + Apply it for FileBasedSpellChecker and IndexBasedSpellChecker both. + (shalin) + +43. SOLR-648: SpellCheckComponent throws NullPointerException on using spellcheck.q request + parameter after restarting Solr, if reload is called but build is not called. + (Jonathan Lee, shalin) + +44. SOLR-598: DebugComponent now always occurs last in the SearchHandler list unless the + components are explicitly declared. (gsingers) + +45. SOLR-676: DataImportHandler should use UpdateRequestProcessor API instead of directly + using UpdateHandler. (shalin) + +46. SOLR-696: Fixed bug in NamedListCodec in regards to serializing Iterable objects. (gsingers) + +47. SOLR-669: snappuler fix for FreeBSD/Darwin (Richard "Trey" Hyde via Otis Gospodnetic) + +48. SOLR-606: Fixed spell check collation offset issue. (Stefan Oestreicher , Geoffrey Young, gsingers) + +49. SOLR-589: Improved handling of badly formated query strings (Sean Timm via Otis Gospodnetic) + +50. SOLR-749: Allow QParser and ValueSourceParsers to be extended with same name (hossman, gsingers) + +51. SOLR-704: DIH NumberFormatTransformer can silently ignore part of the + string while parsing. Now it tries to use the complete string for parsing. + Failure to do so will result in an exception. + (Stefan Oestreicher via shalin) + +52. SOLR-729: DIH Context.getDataSource(String) gives current entity's + DataSource instance regardless of argument. (Noble Paul, shalin) + +53. SOLR-726: DIH: Jdbc Drivers and DataSources fail to load if placed in + multicore sharedLib or core's lib directory. + (Walter Ferrara, Noble Paul, shalin) + +Other Changes + 1. SOLR-135: Moved common classes to org.apache.solr.common and altered the + build scripts to make two jars: apache-solr-1.3.jar and + apache-solr-1.3-common.jar. This common.jar can be used in client code; + It does not have lucene or junit dependencies. The original classes + have been replaced with a @Deprecated extended class and are scheduled + to be removed in a later release. While this change does not affect API + compatibility, it is recommended to update references to these + deprecated classes. (ryan) + + 2. SOLR-268: Tweaks to post.jar so it prints the error message from Solr. + (Brian Whitman via hossman) + + 3. Upgraded to Lucene 2.2.0; June 18, 2007. + + 4. SOLR-215: Static access to SolrCore.getSolrCore() and SolrConfig.config + have been deprecated in order to support multiple loaded cores. + (Henri Biestro via ryan) + + 5. SOLR-367: The create method in all TokenFilter and Tokenizer Factories + provided by Solr now declare their specific return types instead of just + using "TokenStream" (hossman) + + 6. SOLR-396: Hooks add to build system for automatic generation of (stub) + Tokenizer and TokenFilter Factories. + Also: new Factories for all Tokenizers and TokenFilters provided by the + lucene-analyzers-2.2.0.jar -- includes support for German, Chinese, + Russan, Dutch, Greek, Brazilian, Thai, and French. (hossman) + + 7. Upgraded to commons-CSV r609327, which fixes escaping bugs and + introduces new escaping and whitespace handling options to + increase compatibility with different formats. (yonik) + + 8. Upgraded to Lucene 2.3.0; Jan 23, 2008. + + 9. SOLR-451: Changed analysis.jsp to use POST instead of GET, also made the input area a + bit bigger (gsingers) + +10. Upgrade to Lucene 2.3.1 + +11. SOLR-531: Different exit code for rsyncd-start and snappuller if disabled (Thomas Peuss via billa) + +12. SOLR-550: Clarified DocumentBuilder addField javadocs (gsingers) + +13. Upgrade to Lucene 2.3.2 + +14. SOLR-518: Changed luke.xsl to use divs w/css for generating histograms + instead of SVG (Thomas Peuss via hossman) + +15. SOLR-592: Added ShardParams interface and changed several string literals + to references to constants in CommonParams. + (Lars Kotthoff via Otis Gospodnetic) + +16. SOLR-520: Deprecated unused LengthFilter since already core in + Lucene-Java (hossman) + +17. SOLR-645: Refactored SimpleFacetsTest (Lars Kotthoff via hossman) + +18. SOLR-591: Changed Solrj default value for facet.sort to true (Lars Kotthoff via Shalin) + +19. Upgraded to Lucene 2.4-dev (r669476) to support SOLR-572 (gsingers) + +20. SOLR-636: Improve/simplify example configs; and make index.jsp + links more resilient to configs loaded via an InputStream + (Lars Kotthoff, hossman) + +21. SOLR-682: Scripts now support FreeBSD (Richard Trey Hyde via gsingers) + +22. SOLR-489: Added in deprecation comments. (Sean Timm, Lars Kothoff via gsingers) + +23. SOLR-692: Migrated to stable released builds of StAX API 1.0.1 and StAX 1.2.0 (shalin) +24. Upgraded to Lucene 2.4-dev (r686801) (yonik) +25. Upgraded to Lucene 2.4-dev (r688745) 27-Aug-2008 (yonik) +26. Upgraded to Lucene 2.4-dev (r691741) 03-Sep-2008 (yonik) +27. Replaced the StAX reference implementation with the geronimo + StAX API jar, and the Woodstox StAX implementation. (yonik) + +Build + 1. SOLR-411. Changed the names of the Solr JARs to use the defacto standard JAR names based on + project-name-version.jar. This yields, for example: + apache-solr-common-1.3-dev.jar + apache-solr-solrj-1.3-dev.jar + apache-solr-1.3-dev.jar + + 2. SOLR-479: Added clover code coverage targets for committers and the nightly build. Requires + the Clover library, as licensed to Apache and only available privately. To run: + ant -Drun.clover=true clean clover test generate-clover-reports + + 3. SOLR-510: Nightly release includes client sources. (koji) + + 4. SOLR-563: Modified the build process to build contrib projects + (Shalin Shekhar Mangar via Otis Gospodnetic) + + 5. SOLR-673: Modify build file to create javadocs for core, solrj, contrib and "all inclusive" (shalin) + + 6. SOLR-672: Nightly release includes contrib sources. (Jeremy Hinegardner, shalin) + + 7. SOLR-586: Added ant target and POM files for building maven artifacts of the Solr core, common, + client and contrib. The target can publish artifacts with source and javadocs. + (Spencer Crissman, Craig McClanahan, shalin) + +================== Release 1.2 ================== + +Upgrading from Solr 1.1 +------------------------------------- +IMPORTANT UPGRADE NOTE: In a master/slave configuration, all searchers/slaves +should be upgraded before the master! If the master were to be updated +first, the older searchers would not be able to read the new index format. + +Older Apache Solr installations can be upgraded by replacing +the relevant war file with the new version. No changes to configuration +files should be needed. + +This version of Solr contains a new version of Lucene implementing +an updated index format. This version of Solr/Lucene can still read +and update indexes in the older formats, and will convert them to the new +format on the first index change. One change in the new index format +is that all "norms" are kept in a single file, greatly reducing the number +of files per segment. Users of compound file indexes will want to consider +converting to the non-compound format for faster indexing and slightly better +search concurrency. + +The JSON response format for facets has changed to make it easier for +clients to retain sorted order. Use json.nl=map explicitly in clients +to get the old behavior, or add it as a default to the request handler +in solrconfig.xml + +The Lucene based Solr query syntax is slightly more strict. +A ':' in a field value must be escaped or the whole value must be quoted. + +The Solr "Request Handler" framework has been updated in two key ways: +First, if a Request Handler is registered in solrconfig.xml with a name +starting with "/" then it can be accessed using path-based URL, instead of +using the legacy "/select?qt=name" URL structure. Second, the Request +Handler framework has been extended making it possible to write Request +Handlers that process streams of data for doing updates, and there is a +new-style Request Handler for XML updates given the name of "/update" in +the example solrconfig.xml. Existing installations without this "/update" +handler will continue to use the old update servlet and should see no +changes in behavior. For new-style update handlers, errors are now +reflected in the HTTP status code, Content-type checking is more strict, +and the response format has changed and is controllable via the wt +parameter. + + + +Detailed Change List +-------------------- + +New Features + 1. SOLR-82: Default field values can be specified in the schema.xml. + (Ryan McKinley via hossman) + + 2. SOLR-89: Two new TokenFilters with corresponding Factories... + * TrimFilter - Trims leading and trailing whitespace from Tokens + * PatternReplaceFilter - applies a Pattern to each token in the + stream, replacing match occurances with a specified replacement. + (hossman) + + 3. SOLR-91: allow configuration of a limit of the number of searchers + that can be warming in the background. This can be used to avoid + out-of-memory errors, or contention caused by more and more searchers + warming in the background. An error is thrown if the limit specified + by maxWarmingSearchers in solrconfig.xml is exceeded. (yonik) + + 4. SOLR-106: New faceting parameters that allow specification of a + minimum count for returned facets (facet.mincount), paging through facets + (facet.offset, facet.limit), and explicit sorting (facet.sort). + facet.zeros is now deprecated. (yonik) + + 5. SOLR-80: Negative queries are now allowed everywhere. Negative queries + are generated and cached as their positive counterpart, speeding + generation and generally resulting in smaller sets to cache. + Set intersections in SolrIndexSearcher are more efficient, + starting with the smallest positive set, subtracting all negative + sets, then intersecting with all other positive sets. (yonik) + + 6. SOLR-117: Limit a field faceting to constraints with a prefix specified + by facet.prefix or f.<field>.facet.prefix. (yonik) + + 7. SOLR-107: JAVA API: Change NamedList to use Java5 generics + and implement Iterable<Map.Entry> (Ryan McKinley via yonik) + + 8. SOLR-104: Support for "Update Plugins" -- RequestHandlers that want + access to streams of data for doing updates. ContentStreams can come + from the raw POST body, multi-part form data, or remote URLs. + Included in this change is a new SolrDispatchFilter that allows + RequestHandlers registered with names that begin with a "/" to be + accessed using a URL structure based on that name. + (Ryan McKinley via hossman) + + 9. SOLR-126: DirectUpdateHandler2 supports autocommitting after a specified time + (in ms), using <autoCommit><maxTime>10000</maxTime></autoCommit>. + (Ryan McKinley via klaas). + +10. SOLR-116: IndexInfoRequestHandler added. (Erik Hatcher) + +11. SOLR-79: Add system property ${<sys.prop>[:<default>]} substitution for + configuration files loaded, including schema.xml and solrconfig.xml. + (Erik Hatcher with inspiration from Andrew Saar) + +12. SOLR-149: Changes to make Solr more easily embeddable, in addition + to logging which request handler handled each request. + (Ryan McKinley via yonik) + +13. SOLR-86: Added standalone Java-based command-line updater. + (Erik Hatcher via Bertrand Delecretaz) + +14. SOLR-152: DisMaxRequestHandler now supports configurable alternate + behavior when q is not specified. A "q.alt" param can be specified + using SolrQueryParser syntax as a mechanism for specifying what query + the dismax handler should execute if the main user query (q) is blank. + (Ryan McKinley via hossman) + +15. SOLR-158: new "qs" (Query Slop) param for DisMaxRequestHandler + allows for specifying the amount of default slop to use when parsing + explicit phrase queries from the user. + (Adam Hiatt via hossman) + +16. SOLR-81: SpellCheckerRequestHandler that uses the SpellChecker from + the Lucene contrib. + (Otis Gospodnetic and Adam Hiatt) + +17. SOLR-182: allow lazy loading of request handlers on first request. + (Ryan McKinley via yonik) + +18. SOLR-81: More SpellCheckerRequestHandler enhancements, inlcluding + support for relative or absolute directory path configurations, as + well as RAM based directory. (hossman) + +19. SOLR-197: New parameters for input: stream.contentType for specifying + or overriding the content type of input, and stream.file for reading + local files. (Ryan McKinley via yonik) + +20. SOLR-66: CSV data format for document additions and updates. (yonik) + +21. SOLR-184: add echoHandler=true to responseHeader, support echoParams=all + (Ryan McKinley via ehatcher) + +22. SOLR-211: Added a regex PatternTokenizerFactory. This extracts tokens + from the input string using a regex Pattern. (Ryan McKinley) + +23. SOLR-162: Added a "Luke" request handler and other admin helpers. + This exposes the system status through the standard requestHandler + framework. (ryan) + +24. SOLR-212: Added a DirectSolrConnection class. This lets you access + solr using the standard request/response formats, but does not require + an HTTP connection. It is designed for embedded applications. (ryan) + +25. SOLR-204: The request dispatcher (added in SOLR-104) can handle + calls to /select. This offers uniform error handling for /update and + /select. To enable this behavior, you must add: + <requestDispatcher handleSelect="true" > to your solrconfig.xml + See the example solrconfig.xml for details. (ryan) + +26. SOLR-170: StandardRequestHandler now supports a "sort" parameter. + Using the ';' syntax is still supported, but it is recommended to + transition to the new syntax. (ryan) + +27. SOLR-181: The index schema now supports "required" fields. Attempts + to add a document without a required field will fail, returning a + descriptive error message. By default, the uniqueKey field is + a required field. This can be disabled by setting required=false + in schema.xml. (Greg Ludington via ryan) + +28. SOLR-217: Fields configured in the schema to be neither indexed or + stored will now be quietly ignored by Solr when Documents are added. + The example schema has a comment explaining how this can be used to + ignore any "unknown" fields. + (Will Johnson via hossman) + +29. SOLR-227: If schema.xml defines multiple fieldTypes, fields, or + dynamicFields with the same name, a severe error will be logged rather + then quietly continuing. Depending on the <abortOnConfigurationError> + settings, this may halt the server. Likewise, if solrconfig.xml + defines multiple RequestHandlers with the same name it will also add + an error. (ryan) + +30. SOLR-226: Added support for dynamic field as the destination of a + copyField using glob (*) replacement. (ryan) + +31. SOLR-224: Adding a PhoneticFilterFactory that uses apache commons codec + language encoders to build phonetically similar tokens. This currently + supports: DoubleMetaphone, Metaphone, Soundex, and RefinedSoundex (ryan) + +32. SOLR-199: new n-gram tokenizers available via NGramTokenizerFactory + and EdgeNGramTokenizerFactory. (Adam Hiatt via yonik) + +33. SOLR-234: TrimFilter can update the Token's startOffset and endOffset + if updateOffsets="true". By default the Token offsets are unchanged. + (ryan) + +34. SOLR-208: new example_rss.xsl and example_atom.xsl to provide more + examples for people about the Solr XML response format and how they + can transform it to suit different needs. + (Brian Whitman via hossman) + +35. SOLR-249: Deprecated SolrException( int, ... ) constructors in favor + of constructors that takes an ErrorCode enum. This will ensure that + all SolrExceptions use a valid HTTP status code. (ryan) + +36. SOLR-386: Abstracted SolrHighlighter and moved existing implementation + to DefaultSolrHighlighter. Adjusted SolrCore and solrconfig.xml so + that highlighter is configurable via a class attribute. Allows users + to use their own highlighter implementation. (Tricia Williams via klaas) + +Changes in runtime behavior + 1. Highlighting using DisMax will only pick up terms from the main + user query, not boost or filter queries (klaas). + + 2. SOLR-125: Change default of json.nl to flat, change so that + json.nl only affects items where order matters (facet constraint + listings). Fix JSON output bug for null values. Internal JAVA API: + change most uses of NamedList to SimpleOrderedMap. (yonik) + + 3. A new method "getSolrQueryParser" has been added to the IndexSchema + class for retrieving a new SolrQueryParser instance with all options + specified in the schema.xml's <solrQueryParser> block set. The + documentation for the SolrQueryParser constructor and its use of + IndexSchema have also been clarified. + (Erik Hatcher and hossman) + + 4. DisMaxRequestHandler's bq, bf, qf, and pf parameters can now accept + multiple values (klaas). + + 5. Query are re-written before highlighting is performed. This enables + proper highlighting of prefix and wildcard queries (klaas). + + 6. A meaningful exception is raised when attempting to add a doc missing + a unique id if it is declared in the schema and allowDups=false. + (ryan via klaas) + + 7. SOLR-183: Exceptions with error code 400 are raised when + numeric argument parsing fails. RequiredSolrParams class added + to facilitate checking for parameters that must be present. + (Ryan McKinley, J.J. Larrea via yonik) + + 8. SOLR-179: By default, solr will abort after any severe initialization + errors. This behavior can be disabled by setting: + <abortOnConfigurationError>false</abortOnConfigurationError> + in solrconfig.xml (ryan) + + 9. The example solrconfig.xml maps /update to XmlUpdateRequestHandler using + the new request dispatcher (SOLR-104). This requires posted content to + have a valid contentType: curl -H 'Content-type:text/xml; charset=utf-8' + The response format matches that of /select and returns standard error + codes. To enable solr1.1 style /update, do not map "/update" to any + handler in solrconfig.xml (ryan) + +10. SOLR-231: If a charset is not specified in the contentType, + ContentStream.getReader() will use UTF-8 encoding. (ryan) + +11. SOLR-230: More options for post.jar to support stdin, xml on the + commandline, and defering commits. Tutorial modified to take + advantage of these options so there is no need for curl. + (hossman) + +12. SOLR-128: Upgraded Jetty to the latest stable release 6.1.3 (ryan) + +Optimizations + 1. SOLR-114: HashDocSet specific implementations of union() and andNot() + for a 20x performance improvement for those set operations, and a new + hash algorithm speeds up exists() by 10% and intersectionSize() by 8%. + (yonik) + + 2. SOLR-115: Solr now uses BooleanQuery.clauses() instead of + BooleanQuery.getClauses() in any situation where there is no risk of + modifying the original query. + (hossman) + + 3. SOLR-221: Speed up sorted faceting on multivalued fields by ~60% + when the base set consists of a relatively large portion of the + index. (yonik) + + 4. SOLR-221: Added a facet.enum.cache.minDf parameter which avoids + using the filterCache for terms that match few documents, trading + decreased memory usage for increased query time. (yonik) + +Bug Fixes + 1. SOLR-87: Parsing of synonym files did not correctly handle escaped + whitespace such as \r\n\t\b\f. (yonik) + + 2. SOLR-92: DOMUtils.getText (used when parsing config files) did not + work properly with many DOM implementations when dealing with + "Attributes". (Ryan McKinley via hossman) + + 3. SOLR-9,SOLR-99: Tighten up sort specification error checking, throw + exceptions for missing sort specifications or a sort on a non-indexed + field. (Ryan McKinley via yonik) + + 4. SOLR-145: Fix for bug introduced in SOLR-104 where some Exceptions + were being ignored by all "out of the box" RequestHandlers. (hossman) + + 5. SOLR-166: JNDI solr.home code refactoring. SOLR-104 moved + some JNDI related code to the init method of a Servlet Filter - + according to the Servlet Spec, all Filter's should be initialized + prior to initializing any Servlets, but this is not the case in at + least one Servlet Container (Resin). This "bug fix" refactors + this JNDI code so that it should be executed the first time any + attempt is made to use the solr.home dir. + (Ryan McKinley via hossman) + + 6. SOLR-173: Bug fix to SolrDispatchFilter to reduce "too many open + files" problem was that SolrDispatchFilter was not closing requests + when finished. Also modified ResponseWriters to only fetch a Searcher + reference if necessary for writing out DocLists. + (Ryan McKinley via hossman) + + 7. SOLR-168: Fix display positioning of multiple tokens at the same + position in analysis.jsp (yonik) + + 8. SOLR-167: The SynonymFilter sometimes generated incorrect offsets when + multi token synonyms were mached in the source text. (yonik) + + 9. SOLR-188: bin scripts do not support non-default webapp names. Added "-U" + option to specify a full path to the update url, overriding the + "-h" (hostname), "-p" (port) and "-w" (webapp name) parameters. + (Jeff Rodenburg via billa) + +10. SOLR-198: RunExecutableListener always waited for the process to + finish, even when wait="false" was set. (Koji Sekiguchi via yonik) + +11. SOLR-207: Changed distribution scripts to remove recursive find + and avoid use of "find -maxdepth" on platforms where it is not + supported. (yonik) + +12. SOLR-222: Changing writeLockTimeout in solrconfig.xml did not + change the effective timeout. (Koji Sekiguchi via yonik) + +13. Changed the SOLR-104 RequestDispatcher so that /select?qt=xxx can not + access handlers that start with "/". This makes path based authentication + possible for path based request handlers. (ryan) + +14. SOLR-214: Some servlet containers (including Tomcat and Resin) do not + obey the specified charset. Rather then letting the the container handle + it solr now uses the charset from the header contentType to decode posted + content. Using the contentType: "text/xml; charset=utf-8" will force + utf-8 encoding. If you do not specify a contentType, it will use the + platform default. (Koji Sekiguchi via ryan) + +15. SOLR-241: Undefined system properties used in configuration files now + cause a clear message to be logged rather than an obscure exception thrown. + (Koji Sekiguchi via ehatcher) + +Other Changes + 1. Updated to Lucene 2.1 + + 2. Updated to Lucene 2007-05-20_00-04-53 + +================== Release 1.1.0 ================== + +Status +------ +This is the first release since Solr joined the Incubator, and brings many +new features and performance optimizations including highlighting, +faceted browsing, and JSON/Python/Ruby response formats. + + +Upgrading from previous Solr versions +------------------------------------- +Older Apache Solr installations can be upgraded by replacing +the relevant war file with the new version. No changes to configuration +files are needed and the index format has not changed. + +The default version of the Solr XML response syntax has been changed to 2.2. +Behavior can be preserved for those clients not explicitly specifying a +version by adding a default to the request handler in solrconfig.xml + +By default, Solr will no longer use a searcher that has not fully warmed, +and requests will block in the meantime. To change back to the previous +behavior of using a cold searcher in the event there is no other +warm searcher, see the useColdSearcher config item in solrconfig.xml + +The XML response format when adding multiple documents to the collection +in a single <add> command has changed to return a single <result>. + + +Detailed Change List +-------------------- + +New Features + 1. added support for setting Lucene's positionIncrementGap + 2. Admin: new statistics for SolrIndexSearcher + 3. Admin: caches now show config params on stats page + 3. max() function added to FunctionQuery suite + 4. postOptimize hook, mirroring the functionallity of the postCommit hook, + but only called on an index optimize. + 5. Ability to HTTP POST query requests to /select in addition to HTTP-GET + 6. The default search field may now be overridden by requests to the + standard request handler using the df query parameter. (Erik Hatcher) + 7. Added DisMaxRequestHandler and SolrPluginUtils. (Chris Hostetter) + 8. Support for customizing the QueryResponseWriter per request + (Mike Baranczak / SOLR-16 / hossman) + 9. Added KeywordTokenizerFactory (hossman) +10. copyField accepts dynamicfield-like names as the source. + (Darren Erik Vengroff via yonik, SOLR-21) +11. new DocSet.andNot(), DocSet.andNotSize() (yonik) +12. Ability to store term vectors for fields. (Mike Klaas via yonik, SOLR-23) +13. New abstract BufferedTokenStream for people who want to write + Tokenizers or TokenFilters that require arbitrary buffering of the + stream. (SOLR-11 / yonik, hossman) +14. New RemoveDuplicatesToken - useful in situations where + synonyms, stemming, or word-deliminater-ing produce identical tokens at + the same position. (SOLR-11 / yonik, hossman) +15. Added highlighting to SolrPluginUtils and implemented in StandardRequestHandler + and DisMaxRequestHandler (SOLR-24 / Mike Klaas via hossman,yonik) +16. SnowballPorterFilterFactory language is configurable via the "language" + attribute, with the default being "English". (Bertrand Delacretaz via yonik, SOLR-27) +17. ISOLatin1AccentFilterFactory, instantiates ISOLatin1AccentFilter to remove accents. + (Bertrand Delacretaz via yonik, SOLR-28) +18. JSON, Python, Ruby QueryResponseWriters: use wt="json", "python" or "ruby" + (yonik, SOLR-31) +19. Make web admin pages return UTF-8, change Content-type declaration to include a + space between the mime-type and charset (Philip Jacob, SOLR-35) +20. Made query parser default operator configurable via schema.xml: + <solrQueryParser defaultOperator="AND|OR"/> + The default operator remains "OR". +21. JAVA API: new version of SolrIndexSearcher.getDocListAndSet() which takes + flags (Greg Ludington via yonik, SOLR-39) +22. A HyphenatedWordsFilter, a text analysis filter used during indexing to rejoin + words that were hyphenated and split by a newline. (Boris Vitez via yonik, SOLR-41) +23. Added a CompressableField base class which allows fields of derived types to + be compressed using the compress=true setting. The field type also gains the + ability to specify a size threshold at which field data is compressed. + (klaas, SOLR-45) +24. Simple faceted search support for fields (enumerating terms) + and arbitrary queries added to both StandardRequestHandler and + DisMaxRequestHandler. (hossman, SOLR-44) +25. In addition to specifying default RequestHandler params in the + solrconfig.xml, support has been added for configuring values to be + appended to the multi-val request params, as well as for configuring + invariant params that can not overridden in the query. (hossman, SOLR-46) +26. Default operator for query parsing can now be specified with q.op=AND|OR + from the client request, overriding the schema value. (ehatcher) +27. New XSLTResponseWriter does server side XSLT processing of XML Response. + In the process, an init(NamedList) method was added to QueryResponseWriter + which works the same way as SolrRequestHandler. + (Bertrand Delacretaz / SOLR-49 / hossman) +28. json.wrf parameter adds a wrapper-function around the JSON response, + useful in AJAX with dynamic script tags for specifying a JavaScript + callback function. (Bertrand Delacretaz via yonik, SOLR-56) +29. autoCommit can be specified every so many documents added (klaas, SOLR-65) +30. ${solr.home}/lib directory can now be used for specifying "plugin" jars + (hossman, SOLR-68) +31. Support for "Date Math" relative "NOW" when specifying values of a + DateField in a query -- or when adding a document. + (hossman, SOLR-71) +32. useColdSearcher control in solrconfig.xml prevents the first searcher + from being used before it's done warming. This can help prevent + thrashing on startup when multiple requests hit a cold searcher. + The default is "false", preventing use before warm. (yonik, SOLR-77) + +Changes in runtime behavior + 1. classes reorganized into different packages, package names changed to Apache + 2. force read of document stored fields in QuerySenderListener + 3. Solr now looks in ./solr/conf for config, ./solr/data for data + configurable via solr.solr.home system property + 4. Highlighter params changed to be prefixed with "hl."; allow fragmentsize + customization and per-field overrides on many options + (Andrew May via klaas, SOLR-37) + 5. Default param values for DisMaxRequestHandler should now be specified + using a '<lst name="defaults">...</lst>' init param, for backwards + compatability all init prams will be used as defaults if an init param + with that name does not exist. (hossman, SOLR-43) + 6. The DisMaxRequestHandler now supports multiple occurances of the "fq" + param. (hossman, SOLR-44) + 7. FunctionQuery.explain now uses ComplexExplanation to provide more + accurate score explanations when composed in a BooleanQuery. + (hossman, SOLR-25) + 8. Document update handling locking is much sparser, allowing performance gains + through multiple threads. Large commits also might be faster (klaas, SOLR-65) + 9. Lazy field loading can be enabled via a solrconfig directive. This will be faster when + not all stored fields are needed from a document (klaas, SOLR-52) +10. Made admin JSPs return XML and transform them with new XSL stylesheets + (Otis Gospodnetic, SOLR-58) +11. If the "echoParams=explicit" request parameter is set, request parameters are copied + to the output. In an XML output, they appear in new <lst name="params"> list inside + the new <lst name="responseHeader"> element, which replaces the old <responseHeader>. + Adding a version=2.1 parameter to the request produces the old format, for backwards + compatibility (bdelacretaz and yonik, SOLR-59). + +Optimizations + 1. getDocListAndSet can now generate both a DocList and a DocSet from a + single lucene query. + 2. BitDocSet.intersectionSize(HashDocSet) no longer generates an intermediate + set + 3. OpenBitSet completed, replaces BitSet as the implementation for BitDocSet. + Iteration is faster, and BitDocSet.intersectionSize(BitDocSet) and unionSize + is between 3 and 4 times faster. (yonik, SOLR-15) + 4. much faster unionSize when one of the sets is a HashDocSet: O(smaller_set_size) + 5. Optimized getDocSet() for term queries resulting in a 36% speedup of facet.field + queries where DocSets aren't cached (for example, if the number of terms in the field + is larger than the filter cache.) (yonik) + 6. Optimized facet.field faceting by as much as 500 times when the field has + a single token per document (not multiValued & not tokenized) by using the + Lucene FieldCache entry for that field to tally term counts. The first request + utilizing the FieldCache will take longer than subsequent ones. + +Bug Fixes + 1. Fixed delete-by-id for field types who's indexed form is different + from the printable form (mainly sortable numeric types). + 2. Added escaping of attribute values in the XML response (Erik Hatcher) + 3. Added empty extractTerms() to FunctionQuery to enable use in + a MultiSearcher (Yonik) + 4. WordDelimiterFilter sometimes lost token positionIncrement information + 5. Fix reverse sorting for fields were sortMissingFirst=true + (Rob Staveley, yonik) + 6. Worked around a Jetty bug that caused invalid XML responses for fields + containing non ASCII chars. (Bertrand Delacretaz via yonik, SOLR-32) + 7. WordDelimiterFilter can throw exceptions if configured with both + generate and catenate off. (Mike Klaas via yonik, SOLR-34) + 8. Escape '>' in XML output (because ]]> is illegal in CharData) + 9. field boosts weren't being applied and doc boosts were being applied to fields (klaas) +10. Multiple-doc update generates well-formed xml (klaas, SOLR-65) +11. Better parsing of pingQuery from solrconfig.xml (hossman, SOLR-70) +12. Fixed bug with "Distribution" page introduced when Versions were + added to "Info" page (hossman) +13. Fixed HTML escaping issues with user input to analysis.jsp and action.jsp + (hossman, SOLR-74) + +Other Changes + 1. Upgrade to Lucene 2.0 nightly build 2006-06-22, lucene SVN revision 416224, + http://svn.apache.org/viewvc/lucene/java/trunk/CHANGES.txt?view=markup&pathrev=416224 + 2. Modified admin styles to improve display in Internet Explorer (Greg Ludington via billa, SOLR-6) + 3. Upgrade to Lucene 2.0 nightly build 2006-07-15, lucene SVN revision 422302, + 4. Included unique key field name/value (if available) in log message of add (billa, SOLR-18) + 5. Updated to Lucene 2.0 nightly build 2006-09-07, SVN revision 462111 + 6. Added javascript to catch empty query in admin query forms (Tomislav Nakic-Alfirevic via billa, SOLR-48 + 7. blackslash escape * in ssh command used in snappuller for zsh compatibility, SOLR-63 + 8. check solr return code in admin scripts, SOLR-62 + 9. Updated to Lucene 2.0 nightly build 2006-11-15, SVN revision 475069 +10. Removed src/apps containing the legacy "SolrTest" app (hossman, SOLR-3) +11. Simplified index.jsp and form.jsp, primarily by removing/hiding XML + specific params, and adding an option to pick the output type. (hossman) +12. Added new numeric build property "specversion" to allow clean + MANIFEST.MF files (hossman) +13. Added Solr/Lucene versions to "Info" page (hossman) +14. Explicitly set mime-type of .xsl files in web.xml to + application/xslt+xml (hossman) +15. Config parsing should now work useing DOM Level 2 parsers -- Solr + previously relied on getTextContent which is a DOM Level 3 addition + (Alexander Saar via hossman, SOLR-78) + +2006/01/17 Solr open sourced, moves to Apache Incubator |