1/10/2018 - How to Use Flow + Artificial Intelligence to Analyze the News

How to Use Flow + Artificial Intelligence to Analyze the News

In this blog post, I will demonstrate how to build a workflow which extracts and analyzes news articles using artificial intelligence.

The articles we will analyze in this blog post will focus on cryptocurrency. Cryptocurrency markets are currently highly volatile. News heavily influences this volatility and often causes an immediate dramatic impact on the market. Positive news can send the price of a cryptocurrency skyrocketing, whereas negative stories can cause prices to plummet.

Because of this, it is potentially advantageous to aggregate the content of relevant cryptocurrency news articles and analyze the emotions, sentiment, concepts, entities, and keywords across them as a whole. Performing this type of analysis using artificial intelligence could allow us to indicate signals or gain insight that may help us anticipate future market movement.

In this example, I will demonstrate how to use Flow to target and extract all news articles from a website into data. I will show how to leverage Flow's high powered artificial intelligence engine and Turing-complete computational framework to analyze the text of all extracted content.

The output of the cognitive text analysis will be compiled into a dataset for further examination. We will learn how to compute hypercubes across the cognitive output in order to further summarize the data and generate multidimensional views of our results.

I will show how to deploy the workflow to an autonomous agent which can perform the data compilation and analysis process on an ongoing basis and continuously deliver the results.

The workflow we develop will be as follows:

  1. The HTML integration interface will be used to extract and structure all news article links from a target website into an in-memory dataset.
  2. Flow will identify and qualify only valid links to articles using a series of generic expressions.
  3. Flow will loop through the structured dataset of links and dynamically invoke the built-in Watson artificial intelligence functions against each news article.
  4. Flow will pass the news articles to Watson and collect the results of the cognitive analysis. Watson will compute all keywords, emotions, categories, sentiments, named entities, and concepts for each article.

This example will highlight the computational power of the Flow environment. The cognitive workflow that we develop has far-reaching applications beyond just the analysis of the news. The concepts examined in this post will allow anyone to implement advanced ai based analytics against unstructured data quickly.

This example uses the built-in Watson functions. You will need to set up Watson credentials in Flow to perform this yourself. You can learn how to set up these credentials in the blog post here. Watson provides a "Lite" plan which allows up to 30,000 natural language calls per month for free.

If you do not have a Flow account - register here. Flow is free for personal use.

The video below provides the full worked example of the artificial intelligence analysis workflow described above. Check out the video here: