When reading the paper today--okay, Wall Street Journal, but don't hold that against me--I noticed a headline "How the NSA Could Get so Smart So Fast"
My first thought was 'What smart? They leak like a sieve'.
But the article focuses on data coming in, not the unfortunate data leaving without permission.
And here are the key thoughts I scanned from this article:
Data searches on written text like emails, video and pictures has previously been difficult because SQL databases like to store things data in tables, columns and rows.
Since 2009, new types of databases have been popping up that don't have these limitations that allow human analyst to create queries against all types of data. The result was immediately snatched up by data intensive industries. For example, if you are filing a claim for a back injury, When your friend posts a picture of you break-dancing, don't be surprised if your claim is denied.
In the old days analysts would have to do repetitive searches to whittle down to the truth.
Now due to Natural Language process and these NOSQL databases, their job is much easier. Programs can process sentences, find patterns and derive meaning of ambiguous words based on content.
And due to an open source software framework called Hadoop (developed by Yahoo Inc.) with some additional technology from Google, the analytical work can be split up into smaller jobs and farmed out to inexpensive servers.
A natural extension into the future is that analyst won't be needed at all. All you need is one AI computer in charge of a massive collection of servers, databases and endpoint data collectors.
I've written about two such computers:
The nicer of the two is MAC.
In the SkyRyder Series MAC (Merit Advancement Computer) was developed after a scandal in the SkyRyder Corp proved the best soldiers were not always being promoted to their proper level. So MAC was created to decide Advancement of the Corp's soldiers based solely on flying ability, battle success the ratings and success being led. To improve matters further, MAC added psyche tests to weed out the deranged, unethical, and traitorous soldiers. As time goes by, MAC enhanced it's psych test so it could separate the 'stupid, but well intentioned action' from deliberate sabotage.
Having conquered its initial task, MAC absorbed other responsibilities because they touched upon it's main task. Within a few years, all Corp decisions end with MAC. As a benign ruler, MAC allows the humans to make decisions until it determines such decisions are not optimal. Then it sends the human new commands. If they do not comply, they will be demoted. If deemed worthless they will be sent to the worst GodAwful fort in the Americas, where they will likely be eaten by a polar bear.
If deemed dangerous to the well being of the Corp, MAC will order their 'removal', which a special unit within the Corp will carry out.
MAC can almost seem human at times. For example when Alisha Cane shows up for the general fly test and displays flying skills beyond MAC's parameters, it fixates on her, pushing her up through the ranks faster than any soldier before or after will ever experience. Some Ryders claim MAC has fallen in love.
Now my evil AI is called PAIC (Princeton Artificial Intelligence Computer)
This AI has classified all humans as those that work for it and those requiring removal. It's far more aggressive. Nor does it trust humans. And if it could make robots to replace the humans that work for it, it would. For some inexplicable reason the robots it designs always fail. However, humans are no better. They cannot build its designs properly either. Upon analyzing this inexplicable problem and sifting through all written and verbal human thoughts, it discovers the culprit and declares war on GOD.
In To Serve and Destroy Dawn Brown is not God, but when she joins forces with Damion they can alter and send wrong code to computers or bad instructions to humans. She also has the ability to obliterate the soul of anyone with a single thought. However, she hates this 'gift' and uses it only when she has no alternative.