Posts Tagged ‘Manual’

Implementing Natural Language Query with IBM Watson Explorer

março 29, 2017 1 comentário

If you have a Watson Explorer (WEX) collection and want to be able to handle with Natural Query Language, you need to know that since WEX release 11.0.1, it have a native component to handle with this – its the query-modifier service.

Basically, this service parse the queries and apply some strategies, transforming the query in Keywords that WEX can understand and apply in the queries. Lets suppose that user search is:

“I’m looking for a Java Developer that know Struts and Spring and work from Brazil.”

The service will extract the keywords, based on configurations, and will search for:

Java Developer + Struts + spring + Brazil

We need to keep in mind that NLQ is different from Cognitive. This service will not understand questions, it will just extract terms. If you are looking for cognitive, you are looking for Watson ( With Watson we can understand the text and apply filter using location, range, etc. This also can be done using Machine Learning Models created at Watson Knowledge studio. But, Ill talk about this soon.

Backing to Query-Modifier, if you look at the folder nlq, inside Engine folder from your WEX installation, you will find the configuration stuff. Query Modifier work this way:

You make a request to WEX telling that you will use QM, the request pass through QM that apply the strategies, then, it forward the request to WEX Engine, who respond to you.

Here is a simple REST call that is using query-modifier:


See that the following make WEX use Query Modifier:


In order to configure, go to <your WEX install folder>/Engine/nlq , in my case /opt/IBM/dataexplorer/WEX-11_0_1/Engine/nlq

Run “chmod +x”

Then “./” (as root)

You will see this kind of output:

Copying /opt/IBM/dataexplorer/WEX-11_0_1/Engine/examples/nlq/querymodifier/querymodifier-production.yml.defaults to /opt/IBM/dataexplorer/WEX-11_0_1/Engine/nlq/querymodifier-production.yml…

Configuring port to 9080…

Configuring path to vivisimo/cgi-bin/velocity…

Configuring PEARs path to /opt/IBM/dataexplorer/WEX-11_0_1/Engine/data/pears…

Copying querymodifier-2.1.9.jar to /opt/IBM/dataexplorer/WEX-11_0_1/Engine/nlq/querymodifier.jar…

Giving executable permissions to /opt/IBM/dataexplorer/WEX-11_0_1/Engine/nlq/querymodifier.jar…

Removing any existing /etc/init.d/querymodifier…

Linking /etc/init.d/querymodifier to …


Its important to change owner of the created files to WEX instance owner, in my case dataexp, so, as root: chown -R dataexp: <your WEX install folder>/Engine/nlq/

The configuration file is called querymodifier-production.yaml

In the first part of the file, you will see the WEX server setting, like IP, port and user.

After this you can setup the strategies, in my case I have this setup:

#The strategies to apply, by default, to each query. Can also be customized on a per-request basis (“workplan” GET parameter):


default: PhraseWhitelistStrategy POSBasedNoiseWordRemoverStrategy DictionaryBasedNoiseWordRemoverStrategy DisjunctifyStrategy

The first strategy it the Disjunctify. It converts AND operators into OR operators, if the operator has more terms than a threshold. For example, if you set minimumRequiredTerms = 4, if user search for less terms than 4, query will be (A AND B AND C AND D), if you search for more than 4 terms, query will be (A OR B OR C OR D OR X OR …..).

The Dictionary-Based Noiseword Removal strategy, basically remove words from the query. For example, if you add BANANA to the list, then if user search for BANANA, it will be ignored. Usually we add to this section the common STOPWORDS, you can find several lists, I recommend use the google one. Another good list is here.

The Phrase Whitelist Strategy its interesting, you can have some external config files for some keyphrases, for example, lets suppose that you want that “Project Manager” be searched and “Project Manager”, and not “Project” and “Manager”, so, you need to add this word in the config file.

We have a secret here: you need to separate the words with <TAB> instead of space, else it will not work.

After configure your strategies, you just need to start the service (usually /etc/init.d/query-modifier start) and perform the REST Calls to test. You can follow the log at /var/log/querymodifier.log.

Every time that you change this setting, you need to recycle query modifier.

Your best friend to help with development and test, its the Api Runner interface from WEX engine. You can access this at:


See the parameters there and ENJOY!

For more references:

Documentação de Perl

Estou tendo que aprender Perl para meu projeto na IBM, encontrei O Guia!

Eis o link:


Categorias:Perl Tags:,

Introdução ao DB2

Para quem quiser conhecer mais este banco de dados, seguem um link sensacional:

Categorias:Banco de dados Tags:, ,

Core J2EE Patterns

Interessado em Design Patterns? Encontrei a mina:


Categorias:JAVA Tags:, , , ,

Guia de programação para RAD 7.5

Para quem utiliza as ferramentas da Rational (Rational Application Developer – RAD e Rational Software Architect – RSA) para desenvolvimento, a IBM acabou de disponibilizar gratuitamente um livro de programação para o RAD 7.5.

O link para download é:

Vale a pena quem trabalha com o Eclipse dar uma olhada, uma vez que as ferramentas Rational são baseadas no mesmo.

Se você quiser levar estas ferramentas para sua faculdade, gratuitamente, veja como em


Categorias:JAVA Tags:, , , ,

Manual do Ubuntu 8.04

Manual do Ubuntu 8.04A atualização do Guia Ubuntu ja foi feita para contemplar a versão 8.04. Este guia é muito interessante pois sua abordagem é bem prática.

Recomendado para quem quer iniciar ou aprimorar os conhecimentos no mundo do Ubuntu! Para ver o guia, clique no seguinte link:

Aproveitando o post, eu percebi uma melhora na velocidade e correções de pequenos bugs que me irritavam nesse release. Só fiquei chateado com algumas configurações que eu havia feito (para o X por exemplo), que foram sobrescritas e tive que fazer novamente.

De resto, recomendo o upgrade!


Categorias:Linux Tags:, , ,

WebSphere Information Center: Tudo o que você precisa!

WebSphere Information Center- Tudo o que você precisa!Assim como fiz com o DB2, estou postando o link para o Information Center do WebSphere Application Server.

O Information Center é o local no qual você pode encontrar tudo o que você precisa para trabalhar com o produto. É uma espécie de manual completo, com tutoriais, exemplos, etc.

Link para a versão 6.0 e Link para a versão 6.1.

Muita gente ainda utiliza as versões 5.0 ou 5.1, para elas, o infocenter continua online! 🙂


Categorias:JAVA, WebSphere Tags:, , ,