o rPf@snddlZddlZddlmZmZmZmZddlmZddl m Z ddl m Z m Z ddlmZdd lmZmZmZdd lmZdd lmZdd lmZdd lmZdZdZdZdZdZ dZ!dZ"dZ#dZ$dZ%dZ&dZ'dZ(dZ)dZ*dZ+dZ,dZ-d Z.d!Z/d"Z0d#Z1d$Z2d%Z3d&Z4d'Z5d(Z6d)Z7d*Z8d+Z9d,Z:d-Z;d.Zd1Z?d2Z@d3ZAd4ZBd5ZCd6ZDd7ZEd8ZFGd9d:d:ZGGd;d<dd?Z# dd@eee$fd;eeeeee ee!ffdffdAdBZ% dd@eee$fd;eeeee efffdCdDZ&d@eee$ffdEdFZ' dd@eee$fd;eeeee efffdGdHZ(dIed@eee$e)fdJefdKdLZ*  dd@eee$e)fdMed;eeeeee efffdNdOZ+ddPdQZ,dRedSeefdTdUZ-dRedSeefdVdWZ.dRefdXdYZ/dZed[ed\efd]d^Z0dZed\efd_d`Z1daefdbdcZ2ddefdedfZ3ddefdgdhZ4ddefdidjZ5dkdlZ6dmed\e fdndoZ7dmedped\e fdqdrZ8  s  ddmedteduedve dwedxed\ee9fdydzZ:dd{ed|edSeefd}d~Z;ddZ_cursorrr8r9r:r )r&r(r)r3resultr*r*r+_parse_profileXs zSearchCommands._parse_profilecKsri}|dkr|S|D],}t|tr|dkrq t|dkrq |ds"q |dds)q dd|dD||d<q |S)Nrr#rcSsg|] }|d|ddqS)rr )score suggestionr*).0_itemr*r*r+ sz4SearchCommands._parse_spellcheck..r )rAintlen)r&r(r) corrections _correctionr*r*r+_parse_spellcheckgs   z SearchCommands._parse_spellcheckcKs|r dd|DSiS)NcSsi|] }|d|dqS)rr r*)rGkvsr*r*r+ sz4SearchCommands._parse_config_get..r*r;r*r*r+_parse_config_getr@z SearchCommands._parse_config_getc sfddtdtdDS)Ncsi|] }||dqS)r r*)rGir(r*r+rPz1SearchCommands._parse_syndump..rr)rangerKr;r*rSr+_parse_syndumpszSearchCommands._parse_syndumpdcCs|j||dS)zT Create a new batch indexer from the client with a given chunk size ) chunk_size) BatchIndexer)r&rXr*r*r+ batch_indexerszSearchCommands.batch_indexerFNc Cs*t|jg} |dur| |j7} |r| t|dur(t|tr(| t| ||r/| t|r6| t |r=| t | rD| t | rK| t |durkt|t ttfrk| tt|g7} t|dkrk| t |7} | dz| t tjdd|D7} Wnty| |7} Ynw|j| S)a Create the search index. The index must not already exist. ### Parameters: - **fields**: a list of TextField or NumericField objects - **no_term_offsets**: If true, we will not save term offsets in the index - **no_field_flags**: If true, we will not save field flags that allow searching in specific fields - **stopwords**: If not None, we create the index with this custom stopword list. The list can be empty - **max_text_fields**: If true, we will encode indexes as if there were more than 32 text fields which allows you to add additional fields (beyond 32). - **temporary**: Create a lightweight temporary index which will expire after the specified period of inactivity (in seconds). The internal idle timer is reset whenever the index is searched or added to. - **no_highlight**: If true, disabling highlighting support. Also implied by no_term_offsets. - **no_term_frequencies**: If true, we avoid saving the term frequencies in the index. - **skip_initial_scan**: If true, we do not scan and index. For more information see `FT.CREATE `_. NrSCHEMAcs|]}|VqdSr- redis_argsrGfr*r*r+ z.SearchCommands.create_index..) CREATE_CMD index_nameargsappendrrArJrrrrrrlisttuplesetrrK itertoolschain TypeErrorr^execute_command) r&fieldsno_term_offsetsno_field_flags stopwords definitionmax_text_fields temporary no_highlightno_term_frequenciesskip_initial_scanrer*r*r+ create_indexs: (               zSearchCommands.create_indexrncCsXt|jddg}z|ttjdd|D7}Wnty&||7}Ynw|j|S)a Alter the existing search index by adding new fields. The index must already exist. ### Parameters: - **fields**: a list of Field objects to add for the index For more information see `FT.ALTER `_. r[ADDcsr\r-r]r_r*r*r+rarbz2SearchCommands.alter_schema_add..) ALTER_CMDrdrgrjrkrlr^rm)r&rnrer*r*r+alter_schema_adds    zSearchCommands.alter_schema_adddelete_documentscCs|rdnd}|t|j|S)ai Drop the index if it exists. Replaced `drop_index` in RediSearch 2.0. Default behavior was changed to not delete the indexed documents. ### Parameters: - **delete_documents**: If `True`, all documents will be deleted. For more information see `FT.DROPINDEX `_. DD)rm DROPINDEX_CMDrd)r&r| delete_strr*r*r+ dropindexs zSearchCommands.dropindex?c Ks|s| rd}t|j||g} |r| d|dur"| d| ||r7| d|r0| d| r7| d|r?| d|g7} | d | ttj| 7} |durX|j| S|j| S) zS Internal add_document used for both batch and single doc indexing TNOSAVENPAYLOADREPLACEPARTIALNOCREATELANGUAGEFIELDS)ADD_CMDrdrfrgrjrkitemsrm) r&doc_idconnnosaverEpayloadreplacepartiallanguage no_creaternrer*r*r+ _add_documents*          zSearchCommands._add_documentcCsHt|j||g}|r|d|r|d|g7}|dur|j|S|j|S)zX Internal add_document_hash used for both batch and single doc indexing rrN) ADDHASH_CMDrdrfrm)r&rrrErrrer*r*r+_add_document_hash,s    z!SearchCommands._add_document_hashz2.0.0z2deprecated since redisearch 2.0, call hset instead)versionreasonrrrErrrrrc Ks$|j|fd|||||||d| S)a Add a single document to the index. ### Parameters - **doc_id**: the id of the saved document. - **nosave**: if set to true, we just index the document, and don't save a copy of it. This means that searches will just return ids. - **score**: the document ranking, between 0.0 and 1.0 - **payload**: optional inner-index payload we can save for fast i access in scoring functions - **replace**: if True, and the document already is in the index, we perform an update and reindex the document - **partial**: if True, the fields specified will be added to the existing document. This has the added benefit that any fields specified with `no_index` will not be reindexed again. Implies `replace` - **language**: Specify the language used for document tokenization. - **no_create**: if True, the document is only updated and reindexed if it already exists. If the document does not exist, an error will be returned. Implies `replace` - **fields** kwargs dictionary of the document fields to be saved and/or indexed. NOTE: Geo points shoule be encoded as strings of "lon,lat" N)rrrErrrrr)r) r&rrrErrrrrrnr*r*r+ add_document@s+ zSearchCommands.add_documentcCs|j|d|||dS)a Add a hash document to the index. ### Parameters - **doc_id**: the document's id. This has to be an existing HASH key in Redis that will hold the fields the index needs. - **score**: the document ranking, between 0.0 and 1.0 - **replace**: if True, and the document already is in the index, we perform an update and reindex the document - **language**: Specify the language used for document tokenization. N)rrErr)r)r&rrErrr*r*r+add_document_hashxs z SearchCommands.add_document_hashcCs6t|j|g}|r |d|dur|j|S|j|S)a Delete a document from index Returns 1 if the document was deleted, 0 if not ### Parameters - **delete_actual_document**: if set to True, RediSearch also delete the actual document if it is in the index r}N)DEL_CMDrdrfrm)r&rrdelete_actual_documentrer*r*r+delete_documents    zSearchCommands.delete_documentcCsR|j|}dd|D}|}z|d=Wn tyYnwtdd|i|S). Load a single document by id cSi|] \}}t|t|qSr*r rGkvr*r*r+rPrTz0SearchCommands.load_document..idNr*r$hgetallrKeyErrorrr&rrnf2r*r*r+ load_documents   zSearchCommands.load_documentcGs|jt|jg|RS)z Returns the full contents of multiple documents. ### Parameters - **ids**: the ids of the saved documents. )rmMGET_CMDrd)r&idsr*r*r+gets zSearchCommands.getcC|t|j}|t|S) Get info an stats about the the current index, including the number of documents, memory consumption, etc For more information see `FT.INFO `_. rmINFO_CMDrdr,r&r(r*r*r+infos zSearchCommands.info query_paramscCsb|durgSg}t|dkr/|d|t|d|D]\}}||||q |S)Nrparamsr)rKrfr)r&rrekeyvaluer*r*r+get_params_argss    zSearchCommands.get_params_argscCsX|jg}t|tr t|}t|tstdt|||7}|||7}||fS)NzBad query type )rdrAstrr ValueErrortypeget_argsr)r&r3rrer*r*r+_mk_query_argss   zSearchCommands._mk_query_argsr3cCsX|j||d\}}t}|jtg|R}t|tr|S|jt||t|ddS) Search the index for a given query, and return a result of documents ### Parameters - **query**: the search query. Either a text for simple queries with default parameters, or a Query object for complex queries. See RediSearch's documentation on query format For more information see `FT.SEARCH `_. r@@r3r4rtimerm SEARCH_CMDrArr,r&r3rrestr(r*r*r+searchs zSearchCommands.searchcCs$|j||d\}}|jtg|RS)zReturns the execution plan for a complex query. For more information see `FT.EXPLAIN `_. r)rrm EXPLAIN_CMD)r&r3rre query_textr*r*r+explains zSearchCommands.explaincCstd)Nz#EXPLAINCLI will not be implemented.)NotImplementedError)r&r3r*r*r+ explain_cli szSearchCommands.explain_clicCst|trt|j}t|jg|}nt|tr&d}td|jg|}nt d||| |7}|j |}|j t|||dS) Issue an aggregation query. ### Parameters **query**: This can be either an `AggregateRequest`, or a `Cursor` An `AggregateResult` object is returned. You can access the rows from its `rows` property, which will always yield the rows of the result. For more information see `FT.AGGREGATE `_. TREAD Bad queryr3r= rArboolrB AGGREGATE_CMDrd build_argsr CURSOR_CMDrrrmr,r&r3rr=r'rawr*r*r+ aggregate s     zSearchCommands.aggregaterr=cCsz|rt|tr|d|_|}nt|d}|d}nd}t|tr/|jr/|d}|dd}nd}|dd}t|||S)Nr rr)rArcidr _with_schemar)r&rr3r=cursorschemarowsr*r*r+r>-s      z$SearchCommands._get_aggregate_resultlimitedcCst}t|jdg}|r|d|dt|tr&d|d<||7}nt|tr=d|d<||7}|| |7}nt d|j |}|j t||t|dd S) a Performs a search or aggregate command and collects performance information. ### Parameters **query**: This can be either an `AggregateRequest`, `Query` or string. **limited**: If set to True, removes details of reader iterator. **query_params**: Define one or more value parameters. Each parameter has a name and a value. r~LIMITEDQUERY AGGREGATErSEARCHz5Must provide AggregateRequest object or Query object.rr) r PROFILE_CMDrdrfrArrrrrrrmr,)r&r3rrrr'r(r*r*r+profileCs"       zSearchCommands.profilecCs\t|j|g}|r|d|g|r|dd|g|r#|dd|g|j|}|t|S) Issue a spellcheck query ### Parameters **query**: search query. **distance***: the maximal Levenshtein distance for spelling suggestions (default: 1, max: 4). **include**: specifies an inclusion custom dictionary. **exclude**: specifies an exclusion custom dictionary. For more information see `FT.SPELLCHECK `_. DISTANCETERMSINCLUDEEXCLUDESPELLCHECK_CMDrdextendrmr,r&r3distanceincludeexcluder'r(r*r*r+ spellcheckks   zSearchCommands.spellchecknametermscGt|g}|||j|S)zAdds terms to a dictionary. ### Parameters - **name**: Dictionary name. - **terms**: List of items for adding to the dictionary. For more information see `FT.DICTADD `_. ) DICT_ADD_CMDrrmr&rrr'r*r*r+dict_add  zSearchCommands.dict_addcGr)aDeletes terms from a dictionary. ### Parameters - **name**: Dictionary name. - **terms**: List of items for removing from the dictionary. For more information see `FT.DICTDEL `_. ) DICT_DEL_CMDrrmrr*r*r+dict_delrzSearchCommands.dict_delcCst|g}|j|S)zDumps all terms in the given dictionary. ### Parameters - **name**: Dictionary name. For more information see `FT.DICTDUMP `_. ) DICT_DUMP_CMDrm)r&rr'r*r*r+ dict_dumps zSearchCommands.dict_dumpoptionrreturncCstd||g}|j|}|dkS)Set runtime configuration option. ### Parameters - **option**: the name of the configuration option. - **value**: a value for the configuration option. For more information see `FT.CONFIG SET `_. SETOK CONFIG_CMDrmr&rrr'rr*r*r+ config_sets zSearchCommands.config_setcCs td|g}|j|}|t|S)Get runtime configuration option value. ### Parameters - **option**: the name of the configuration option. For more information see `FT.CONFIG GET `_. GETrrmr,r&rr'r(r*r*r+ config_gets  zSearchCommands.config_gettagfieldcCs|t|j|S)z Return a list of all possible tag values ### Parameters - **tagfield**: Tag field name For more information see `FT.TAGVALS `_. )rm TAGVALS_CMDrd)r&r r*r*r+tagvals zSearchCommands.tagvalsaliascC|t||jS)z Alias a search index - will fail if alias already exists ### Parameters - **alias**: Name of the alias to create For more information see `FT.ALIASADD `_. )rm ALIAS_ADD_CMDrdr&rr*r*r+aliasaddr zSearchCommands.aliasaddcCr)z Updates an alias - will fail if alias does not already exist ### Parameters - **alias**: Name of the alias to create For more information see `FT.ALIASUPDATE `_. )rmALIAS_UPDATE_CMDrdrr*r*r+ aliasupdater zSearchCommands.aliasupdatecC |t|S)z Removes an alias to a search index ### Parameters - **alias**: Name of the alias to delete For more information see `FT.ALIASDEL `_. )rm ALIAS_DEL_CMDrr*r*r+aliasdels zSearchCommands.aliasdelcOsl|jdd}|D]'}t||j|jg}|dr|d|jr*|d||j|j|q|dS)aJ Add suggestion terms to the AutoCompleter engine. Each suggestion has a score and string. If kwargs["increment"] is true and the terms are already in the server's dictionary, we increment their scores. For more information see `FT.SUGADD `_. F transaction incrementINCRr pipelineSUGADD_COMMANDstringrErrfrrmexecuter&r suggestionsr)pipesugrer*r*r+sugadds       zSearchCommands.sugaddrcCr)z Return the number of entries in the AutoCompleter index. For more information see `FT.SUGLEN `_. )rmSUGLEN_COMMAND)r&rr*r*r+suglens zSearchCommands.suglenr cCs|t||S)z Delete a string from the AutoCompleter index. Returns 1 if the string was found and deleted, 0 otherwise. For more information see `FT.SUGDEL `_. )rmSUGDEL_COMMAND)r&rr r*r*r+sugdelszSearchCommands.sugdel prefixfuzzynumr7 with_payloadsc Csht||d|g}|r|t|r|t|r|t|j|}g} |s'| St|||} dd| DS)t Get a list of suggestions from the AutoCompleter, for a given prefix. Parameters: prefix : str The prefix we are searching. **Must be valid ascii or utf-8** fuzzy : bool If set to true, the prefix search is done in fuzzy mode. **NOTE**: Running fuzzy searches on short (<3 letters) prefixes can be very slow, and even scan the entire index. with_scores : bool If set to true, we also return the (refactored) score of each suggestion. This is normally not needed, and is NOT the original score inserted into the index. with_payloads : bool Return suggestion payloads num : int The maximum number of results we return. Note that we might return less. The algorithm trims irrelevant suggestions. Returns: list: A list of Suggestion objects. If with_scores was False, the score of all suggestions is 1. For more information see `FT.SUGGET `_. MAXcSg|]}|qSr*r*rGsr*r*r+rI[z)SearchCommands.sugget..SUGGET_COMMANDrfrrr rmr) r&rr,r-r.r7r/rer(resultsparserr*r*r+sugget%s(     zSearchCommands.suggetgroupid skipinitialcGs0t|j|g}|r|dg|||j|S)a Updates a synonym group. The command is used to create or update a synonym group with additional terms. Only documents which were indexed after the update will be affected. Parameters: groupid : Synonym group id. skipinitial : bool If set to true, we do not scan and index. terms : The terms. For more information see `FT.SYNUPDATE `_. r) SYNUPDATE_CMDrdrrm)r&r;r<rr'r*r*r+ synupdate]s    zSearchCommands.synupdatecCr)a Dumps the contents of a synonym group. The command is used to dump the synonyms data structure. Returns a list of synonym terms and their synonym group ids. For more information see `FT.SYNDUMP `_. )rm SYNDUMP_CMDrdr,rr*r*r+syndumpus zSearchCommands.syndump)rW) FFNNFNFFF)F)NFrNFFNF)NrNF)FrNFFNF)rNF)NFr-)FNNNNFr+FF)=__name__ __module__ __qualname____doc__r,r2r<r?rDrNrQrVrZrxrrr{rrrrrfloatrrrrrrrrrrJbytesrrrrrrrrr>rrrrrrr r rrrr&r(r*rr:r>r@r*r*r*r+r!?s$ %   G +   5                  (        8 r!c@seZdZddZ d!deeefdeeeeee fffddZ d!deeefdeeeeee fffdd Z d"d d Z d ed ede fddZd edefddZddZddZ    d#dedede dede de deefdd ZdS)$AsyncSearchCommandscs"|t|jIdH}|t|S)rNrrr*r*r+rs zAsyncSearchCommands.infoNr3rcs`|j||d\}}t}|jtg|RIdH}t|tr!|S|jt||t|ddS)rrNrrrrr*r*r+rs zAsyncSearchCommands.searchcst|trt|j}t|jg|}nt|tr'd}td|jg|}nt d||| |7}|j |IdH}|j t|||dS)rTrrNrrrr*r*r+rs    zAsyncSearchCommands.aggregatecsdt|j|g}|r|d|g|r|dd|g|r$|dd|g|j|IdH}|t|S)rrrrrNrrr*r*r+rs  zAsyncSearchCommands.spellcheckrrrcs&td||g}|j|IdH}|dkS)rrNrrrr*r*r+rs zAsyncSearchCommands.config_setcs,td|g}i}|j|IdH}|t|S)rrNrrr*r*r+r s   zAsyncSearchCommands.config_getcsZ|j|IdH}dd|D}|}z|d=Wn ty#Ynwtdd|i|S)rNcSrr*r rr*r*r+rPrTz5AsyncSearchCommands.load_document..rr*rrr*r*r+rs  z!AsyncSearchCommands.load_documentcst|jdd}|D]'}t||j|jg}|dr|d|jr+|d||j|j|q |IdHdS)aI Add suggestion terms to the AutoCompleter engine. Each suggestion has a score and string. If kwargs["increment"] is true and the terms are already in the server's dictionary, we increment their scores. For more information see `FT.SUGADD `_. FrrrrNrrr"r*r*r+r&s      zAsyncSearchCommands.sugaddFr+rr,r-r.r7r/c spt||d|g}|r|t|r|t|r|t|j|IdH}g} |s+| St|||} dd| DS)r0r1NcSr2r*r*r3r*r*r+rI]r5z.AsyncSearchCommands.sugget..r6) r&rr,r-r.r7r/reretr8r9r*r*r+r:'s(    zAsyncSearchCommands.suggetr-rArB)rCrDrErrrrrrJrGrrrrrr rr&rrr:r*r*r*r+rIsL    rI)Irjrtypingrrrr redis.clientr redis.utilsrhelpersr r _utilr aggregationrrrdocumentrr3rrCrrFrrrcrzrrrDROP_CMDrrEXPLAINCLI_CMDrrrrrrrrGET_CMDrrr rrrrrr)r'r7r=r?rrrrrrrrrrr r!rIr*r*r*r+s~       I