All Things Techie With Huge, Unstructured, Intuitive Leaps

Impressed with Microsoft -- finally -- Azure

I, like a lot of other geeks, have become greatly disillusioned with Microsoft in the past several years. I saw them as anti-innovative, fat-cats protected a revenue stream that did not favors for its users, and becoming a stodgy, quaint grandparent in a tech world, where it thought that it was still the same sex object that it was in its early daze.

Microsoft, in my opinion has hung on too long to its archaic operating system which is essentially one big kludge onto top of a stack of turtles of kludges all the way down to the bare silicon. All of their innovations in almost every endeavor from tablets to phones, to music services,  have been market failures because they stubbornly resisted changes to their bloated, digital-cholesterol clogged operating system.  If they truly want to be innovative, they would ditch it in favor of a brand of QNX or Linux for a sleek, less vulnerable system.  Back in the early daze of the 8086 microprocessor, I saw a QNX system being able to boot from one floppy disk, and in its day, that was amazing.

Now that I got that off my chest, I must grudgingly admit that Microsoft has lit a spark that impresses me with their Azure big data suite.  If they are going to re-invent themselves and breathe new life back into the corporation and become innovative again, then Azure might be the vehicle.

Big Data is where it is at, and where it is going to be if we want to manage and monetize the Internet of Everything. And Microsoft Azure is trying to create and promulgate products to that end with Azure.  I only became aware of Azure when several members of the Azure team followed me on Twitter, and when I checked them out, I realized that it wasn't Bill Gates' Microsoft. I really liked what I saw.

Azure offers data analysis as a service, and they have a free component. It is done in a quasi-cloud environment, and from what I see, once you graduate from the newbie class, the prices is okay.  The good news is that there is a link to some pretty nifty free tools.  Here is the link:

The tools are varied, useful and intriguing.

Microsoft just may have a chance to dominate the market.  Their thin edge of the wedge with azure is great, but they must follow the template of Microsoft Word when it started to dominate the marketplace.  Back in the day, personal computers were useful, but not that useful when it came to creating documents electronically.  The IBM Selectric typewriter was the weapon of choice to use up reams of paper.  Then along came the word processor.  Dr. An Wang made a fortune from inventing computer memory, and then sunk his money into Wang Labs headquartered in Lowell Massachusetts. The Wang word processor became ubiquitous for several years. It was a dedicated piece of hardware, and tightly coupled software that didn't do anything else except create formatted documents.  Prior to that, electronic documents were printed on a dot matrix or impact printer without stylings.  (The Wang OS was the first OS that I successfully hacked).

Microsoft Word came out and essentially destroyed the word processor.  It was order of magnitude cheaper, easier to use and a mere fraction of the cost.  It is still the dominant document creator to this day.  Microsoft needs to do the same thing with Azure.

Right now, a lot of the Azure products use the statistical language R. Other plugins calculate linear regression, and all sorts of stuff like standard deviation, blah blah blah.  Microsoft needs to hide that under a big layer of abstraction and make all of that invisible to the end user. Picture the end user who runs a niche cafe in a hip town. Their Point-Of-Sale and computer system collects metrics, meta-data and machine data.  The owners of this data has no idea what this data can tell them or how they can increase their revenue streams.  They don't know Bayesian inference from degree of confidence.

Microsoft needs to build data analysis for the common person, like they built word processing for the common person.  If they do that, they will take their company into the next century. If not, they will be the biggest Edsel of the tech industry.  However, for the first time in a long time, I like what I am seeing come out of Microsoft.


I love ACID.  In the database sense. To remind you what ACID means, here is an excerpt from Wikipedia:

ACID (Atomicity, Consistency, Isolation, Durability) is a set of properties that guarantee that database transactions are processed reliably.


Main article: Atomicity (database systems)
Atomicity requires that each transaction be "all or nothing": if one part of the transaction fails, the entire transaction fails, and the database state is left unchanged. An atomic system must guarantee atomicity in each and every situation, including power failures, errors, and crashes. To the outside world, a committed transaction appears (by its effects on the database) to be indivisible ("atomic"), and an aborted transaction does not happen.


The consistency property ensures that any transaction will bring the database from one valid state to another. Any data written to the database must be valid according to all defined rules, including constraints, cascades, triggers, and any combination thereof. This does not guarantee correctness of the transaction in all ways the application programmer might have wanted (that is the responsibility of application-level code) but merely that any programming errors cannot result in the violation of any defined rules.


The isolation property ensures that the concurrent execution of transactions result in a system state that would be obtained if transactions were executed serially, i.e. one after the other. Providing isolation is the main goal of concurrency control. Depending on concurrency control method, the effects of an incomplete transaction might not even be visible to another transaction.


Durability means that once a transaction has been committed, it will remain so, even in the event of power loss, crashes, or errors. In a relational database, for instance, once a group of SQL statements execute, the results need to be stored permanently (even if the database crashes immediately thereafter). To defend against power loss, transactions (or their effects) must be recorded in a non-volatile memory.

Why bring this up?  Recently I was playing with some feed-forward, backwards propagating Multi-Layer Perceptrons.  I love them because they allow machines to make decisions.  However the decision-making process is not ACID in the context above, if your artificial neural network composed of multi-layer perceptrons is in the continuous learn mode (each successive new bit of information is back propagated with a learning rate).

This may be good or bad, depending on the state of the fuzziness of what had to be decided by the machine, but wouldn't it be nice to have ACID in an artificial neural network?

Get your thinking caps on, and leave me some comments to inspire my imagination and perhaps create a whole new breed of technology.

The Death of the On / Off Switch

I was talking to a colleague of mine and we were discussing, of all things, Bugs Bunny cartoons. In a couple of cartoons, Bugs is in a cabin with Yosemite Sam, and in one instance, there is an old style nickel plated wood stove and in another instance, there is a pot belly stove.  If you don't know what a pot belly stove looks like, here is one:

The conversation revolved around how Bugs Bunny is still popular with today's kids, and yet it was first created in 1940.  My colleague idly wondered if the kids of today know what a pot belly stove was, how it was used and what it was for.  If you are a certain age, and are a grandparent, you may remember them from cabins or cottages, but I doubt that anyone younger than 40 years old has seen one.  So the question is, can the kids of today relate to the cultural load of the concept of a pot belly stove?

The reason that I bring this up, is because the On / Off switch is going the way of the pot belly stove.  Back in the good old, primitive days, an On / Off switch was fairly mechanical. It consisted of a toggle and two contacts separated by a space. When you flipped the switch, the juice flowed and when you turned it off, the contact was broken and the current stopped flowing.

This concept carried on for a long time.  But the On / Off switch is going the way of the dodo bird. Back in the day BR (before remote), you walked over to a TV and pulled a switch to turn it on. The earliest desktop computers had a toggle switch somewhere on the side. A switch was a simple  concept to learn.  Not so any more.

Televisions for example, do not go dead when you switch them off. They have a quiescent circuit just listening for a remote signal. To turn on my iPad, you press a button and to make it work, you do a swipe on the face of it.  On my car, I no longer have to turn on headlights. When it gets dark, they turn on themselves.

With Moore's Law, where transistors are getting smaller and smaller, and smarts are built into everything, the On / Off switch is dying.  Devices will have sensors that will detect when power is needed to the main circuits and they will switch them from solid state switches or transistors. Vacuum cleaners will have motion sensors, and they will know when to turn on the sucking mechanism.  The list goes on and on. New cars detect when a key is in the pocket of the driver and you push a button. No more mechanical linkages.  I am reminded of an 80-some year old man who won an iPod at a church bazaar and didn't know what it was.  He had no inkling that it could be turned on and something so small could play music.

So one day in the future, a child will see some sort of image on an On /Off switch and wonder what it is for and why it was needed.  They will be flummoxed by it.  It's a Brave New World that is coming.  Me -- I say "Bring it on!"

Process Mining, OpenStack and Possibly a New Java Framework

In my process data mining course, on the internal forums,  an OpenStack developer asked how the event logs from using OpenStack could be used in process mining.  This is how I replied:

First of all, let me congratulate you on OpenStack.  I am both a user, and I use the services of an OpenStack driven Platform-As-A-Service to host the development of my mobile apps.

I would see several potentially huge benefits if you incorporated process mining into the OpenStack platform.  For example. spammers now use virtual OpenStack concept to set up a virtual machine, do their spamming or hacking and then tear down the machine never to be seen again.  If you got a signature or a process of this activity, you could theoretically intercept it while it is happening.

Another possibility, is that every time the software does a create, an instantiation, an instant of a virtual, or anything, if you record the timestamps of these machine events, you could provide a QoS or quality of service metric, both for monitoring the cloud and for detecting limitations caused by hardware, software or middleware bottlenecks.

I can see a possibility for mis-configuration of parameters that degrade service quality, that would be picked up by a process mining that would detect missing setup steps in the process.  In other words, an arc around a required region would indicate that required steps were missing.

This course has inspired me to start working on a Java framework (maybe a PROM plugin) that operates on an independent thread (maybe in an OpenStack incarnation) that monitors activity on a server and compares it to ideal processes in real time and flags someone if a crucial process deviates from it.  I think that I could get this going in a timely fashion.

Once again, this course has opened my eyes to potential methodologies and algorithms that can be applied to non-traditional fields.

Note: PROM is an open source process mining tool. The data mining course is given by the Eindhoven University of Technology in the Netherlands.

Process Mining, Data Mining, Explicit & Implicit Events

The course in process data mining given by Professor Wil van der Aalst from the Eindhoven University of Technology in the Netherlands, has opened my eyes to a few elements in data mining that I had not considered.

At first blush, the course looks like it would be quite useful for determining bottle necks in processes like order fulfillment, or patient treatment in a hospital, service calls or a manufacturing environment, and it is. But to an eCommerce platform builder like myself, it can provide amazing insights that I had never thought of before taking this course.

Professor van der Aalst has introduced a layer of abstraction or perhaps a double layer of abstraction in defining any process with a Petri Net derived from an event log. Here is an example of a Petri Net (taken from Wikipedia) :

The P's are places and the T's are transitions. In the theoretical and abstract model, tokens (the black dots) mark various spots in the process. Tokens are consumed by transitions, and regenerated when they arrive at the next occurring place.  The arrival of a token at a specific place, records an explicit behavior in the transition. So how did this help me?

I do  data mining to enhance revenue stream on our eCommerce platform. (See the blog entry below this one).  Previous data mining efforts on my part dealt with implicit events. Sure we had an event log, but we looked at the final event of say a customer purchasing something, and tried to find associations that drove the purchase (attributes or resources like price, color, time of day, past buys of the customer, etc).   The customer's act of making the purchase was captured in the event logs, using timestamps of various navigations, but all of the events leading to purchased were implicit events that we never measured.  With the event logs, we have explicit behaviors, and using those event logs, we can define the purchase process for each customer.  So we start making process maps of the online events that led to the purchase. In short, we began to look at the explicit events.

Where will this take us? It will show us the activities and processes leading to a high value event for us (a purchase).  What it does, is that we isolate high value process events, and by mapping customer behavior to those events, we can evaluate and refine which customers will end up making an online purchase.  So we can treat those customers in a special way with kid gloves.

In essence, we can gain insight into the probability of an online purchase if a new customer starts creating events in our event logs, which indicates behavior that leads to a purchase. This data is extremely valuable, as now we can put this customer on our valued customer list, and using other data mining techniques, we can suggest other things that the customer is interested in and get more sales.

To recap, we now can measure explicit behaviors instead of implicit behaviors based on such limited metrics as past buying behaviors. We add a whole new dimension in enhancing the shopping experience for our users, and thereby enhancing our bottom line revenue stream.

As in life, often in data mining, it pays to pay attention to the explicit things.  Process mining is an incredibly efficient way to deduce explicit behaviors that lead to desired outcomes on our platforms.

Process Mining & Me

Image copyright by Professor Wil van der Aalst, Eindhoven University of Technology

I am taking a course on Process Mining from the Eindhoven University of Technology, and I recently posted this on discussion forum:

I am taking process mining to a different arena, using the basic methodology and event logs. I understand the necessity for well defined proces
ses in relation to things like ISO 9001 and quality management and the achievement of Six Sigma. I started my career as a circuit designer in a military electronics shop with a major designer/manufacturer and not only did we have to have incredibly good yields from the fab shop, but we had to have reliable equipment that we sold to NATO forces.  Process improvement involved saving time, money and resources and creating optimal performance.

However I have moved on and now I am using process discovery in the opposite sense -- in e-Commerce to improve revenue streams.  Essentially, we have a captive platform where self-identified industry insiders buy and sell to each other on a wholesale level. Our platform has several areas where our clients spend time. They can create trusted buyer zones with their circle of buyers and sellers (platform enabled geo-location). They can create packages and offer them for sale to platform escalated groups. They can invoke software robots to buy and sell for them. They can offer and buy and sell from classified listings.  In short, we want to map the processes of how our customers use our platform, and hence optimize the UIX or User Interface Experience to maximize revenue.

We have event logs and timestamps for everything, from when they log in, to when they change their buyer/seller groups, to when they consign inventory, or make offers and counter offers, or do browse the listings.  However the event logs and time stamps are not located in one database table. The challenge was to create an effective case id to tie the disparate event logs together.  Luckily our platform is based on Java Classes, Java Server Pages, Facelets, Taglets and the whole J2EE environment.  As a result, we have a discrete session which is serializable and by simply altering all the event logs by caching the system-generated session id keyed to each event in the disparate event logs, we will have created a powerful customer analysis tool on our platform.

This will enable us to take things a step further. You have heard of responsive UIX designs to adapt to whatever device is utilizing the platform. The process discovery outlined above with enable us to push the boundaries of responsive design to create a machine-customized UIX that facilitates customer behavior on our platform to maximize revenue stream.  Each customer will have a process map based on past behavior, and that process map will generate the UIX with a custom menu, that will be different for each customer types.

Our previous datamining looked for relationships between product groups and buying behavior. It looked at time-domain factors and essentially all sorts of data-dimensions and Bayesian inference of the interrelationships between those dimensions, to enhance revenue stream.

I realize that this doesn't exactly fit into the accepted semantics of a what a process is in the context of this course, but in a larger sense, we are discovering the buying process or the behavior processes on a trade platform that leads to facilitating buying behavior in our users. It adds event processing to our data mining, and this is where this course adds value for me.

iOS & Xcode -- couldn't upload app to iTunes Connect

So, the time finally came to upload my app to iTunes Connect through Xcode.  I went through the Organizer and pressed the submit button, and the frigging thing got stuck.  It would go past "Authenticating With The iTunes Store" stage. It just sat there. Sat there longer than an hour. It was just hugely constipated.  Nothing. Nada.

Realized that the https secure upload was somehow failing. To fix it, I opened a Terminal window and navigated to:


I opened up a vi text editor and changed this line:

# https.proxyPort=443  
to this line:

Worked like a charm after that. All that our product manager would say, was "Crapple", but he is biased.

Adventures in Speculative Software Engineering ~ How A Database Can Control Things

(click on pic for larger image)

The idea struck me as to how a regular, ordinary database can be a controller like a thermostat, or in essence any other kind of feedback control. Since this is speculative engineering, all of the components do not have to exist at this time.  This is presented in the hope that somebody gets a brainstorm from reading this, and makes something like this real.  So here goes:

You would have a sensor to start things off.  Let's suppose that it is a temperature sensor.  It would be a simple thermocouple that is made to measure temperature.  You would need an analog to digital converter so that it could be fed into a digital compute system.  Notice I didn't say computer, but compute.

Then you would need an interface to make sense of what the sensor is telling you after the Analog-to-Digital Conversion.  For that, I propose my handy-dandy XML based Universal Sensor Transfer Protocol, but instead of XML it is STML or Sensor Transfer Markup Language.  Here is what it looks like:

<?stml version="1.0" encoding="utf-8"?>
      <name>Caliente Temp Sensor</name>

Simple. So the STML package arrives at the server. The server read function reads the STML stream, converts the byte array to an object that coincides with the STML taxonomy.  This gets wordified into an NLP (Natural Language Processing) syntax that is deconstructed by a database stored procedure.  The database records the reading (temperature in this case) and a function is called to evaluate the temperature reading.  A table is read as to what the temperature should be at this time of day. and the delta is also recording.  As the delta (ideal temp minus actual temp) is poked into the database, a trigger is fired to create the appropriate response, that is fed back to the server.

The server encodes it into another control markup language that fires say the furnace or air conditioning, depending on the reading.  This control markup language is converted to machine language in an digital to analog converter and the appropriate control signal is applied to the device.

So why would have this instead of a $75 thermostat.  Firstly, if the components were developed to be off-the-shelf, you could hack an entire house to be run off a laptop with portion of it acting as a server.  You wouldn't need custom thermostats or devices, just a bunch of cheap sensors that can talk to anything.  The database will automagically record everything, and you will have your own Big Data stack where another program on your laptop/server will mine the data and extract how things should be run autonomously based on that data,

You now can look for inferences in the data between different things. For example, if you manually turn down the heat at night, or turn up the AC at night, after a few cycles, the machine learns the drill and does it for you.

A component based sensory system like this, ending up in a database has unlimited applications in making your life easier, cheaper, greener and hassle-free. I will race you to the patent office.

Why I don't like mobile frameworks like Cordova and Phonegap

Here is why I don't like cross-platform mobile app development frameworks based on Javascript.  Google is threatening to take down our app because of a vulnerability.  Here is their email to us:

Subject: Security Alert: Apache Cordova vulnerabilities in your Google Play app

This is a notification that your app, is built on a version of Apache Cordova that contains security vulnerabilities. This includes a high severity cross-application scripting (XAS) vulnerability. Under certain circumstances, vulnerable apps could be remotely exploited to steal sensitive information, such as user login credentials.

You should upgrade to Apache Cordova 3.5.1 or higher as soon as possible. For more information about the vulnerabilities, and for guidance on upgrading Apache Cordova, please see

Please note, applications with vulnerabilities that expose users to risk of compromise may be considered “dangerous products” and subject to removal from Google Play.


Google Play Team

So, our app will be considered a dangerous product.  We are scrambling to upgrade.  You don't get these issues with native Java or XCode Objective C.

New MacBook Pro

I just unboxed a new MacBook Pro.  Interesting to see that Apple thinks that the iCloud totally replaces a CD/DVD drive.  I wonder what the automagic signing up of me to the iCloud when I setup up the machine for the first time, does to my privacy.  Just saying ....

Concepts to Cogitate

(click on pic to get the whole nine yards)

I started working with graph databases this morning.  Life is sometimes a graph.  (  It struck me that a graph database can describe complex and fuzzy relationships.  Here is the rub though -- it takes a human to interpret these fuzzy relationships.  A graph database can store them and retrieve them and make machine inferences using them, but it really can't understand them.

Scroll down to the blog entry below on Machine Consciousness and you will get the drift of what I am talking about.  I met my friend Al at the venue pictured above, and he asked a cogent, rhetorical question:

"At what point does logic become intelligence?"

It was an amazing insight into the foundations of intelligence and quite a cogent one.  Without logic, there is no intelligence.  It is almost a precursor.  To a sophisticated state machine, there is lots of logic. What is required to make the leap to intelligence?  Interesting, and to be continued.

Java - determine an image or file exists

One of our programmers asked me how to determine if an image or a file exists last week, so I thought that I would post this code snippet for anyone else that needs a reminder.

String thumbnail = <put the name of image or file here>
   File iFile = new File(config.getServletContext().getRealPath("/")
    + "albums/" + thumbnail);  //we keep all of our images in a directory called albums
   if (iFile.exists()) {
    String thisImage = "albums/"+ thumbnail;
     }  else {
        String thisImage = "albums/"+ "noImage.jgp";
catch (Exception e)
  System.out.println("Can't find image.");

Hope this helps someone.

Musings on Machine Consciousness or Artificial Consciousness

(click on photograph to enlarge)

I was walking along a country road and I came upon this scene picture above.  The picture really does no justice to the rolling hils, the lines, folds and textures as well as the light and dark areas.  As you eye takes in the panorama in real life, you realize that it is a magical place that makes you feel numinous.

And right in front of me, was a guy with a huge piece of paper, trying to capture the scene with a charcoal sketch in preparation for a painting.  The back of his VW stationwagon was lifted open and sketches were scattered all over the car.  I peeked at his sketches, and we started talking.

As it turned out, he was from Australia,  an ex-computer engineer, a mainframe guy who was now an artist and had been for over 20 years.  He earns his living with his art.

We got to talking about stuff, computers and such, and the magnificent scene in front of us, and the impossibility of reproducing and how it made us both feel numinous and from there we got on to consciousness.

I brought up the topic that Ray Kurzweil had predicted a machine with a soul with his book "The Age of A Spiritual Machine".  My new-friend begged to differ, citing that we really didn't know what consciousness was, so how could we emulate or even create it on doped silicon.

I wanted to beg to differ with the artist.  My own neural nets tell me that consciousness is an over-developed tropism.  Plants started it with phototropism for the leaves and geotropism for the roots, and after several hundred million years of evolution gone wild, the tropisms developed into senses, the brain evolved to process the sensory stimuli and it didn't stop.  We developed the ability to abstract, to think, and to integrate the sensory data to knowledge and preserve and pass it on to our species.

This got me to thinking about how a primitive artificial consciousness would be formulated in a computer.  We already have the prototypes with QoS circuits in servers monitored Quality of Service.  We have a heartbeat, plus all sorts of monitors that read and report on processes, cycles, peripherals, services and such.  Right now, these QoS circuits report to humans.  They would have to report to the machine itself.  And the machine would have to react to them.

How it reacts is the tricky part.  In humans, we react to our sensory input by abstracting it to a higher plane.  If for example, our heart skips a beat, we start worrying about a heart attack.  What it means for machine consciousness, is that the data integration into information and finally stringing information into knowledge has to be developed.  As a further step, knowledge must be abstracted into an ideal that can be applied to many other things.  For example, any child knows the following ideal:

Using this ideal, a child can match the above to this:

And with even a bit of cogitation, a child can match the abstract with this:

So, just to recap to this point, a computer would need sensory inputs from everything about itself, hardware and software, and then it would need some powerful inference tools to sort the Big Data from its sensors into information, and from there in knowledge and from there, into an abstract ideal, or an interface if you will, in programming language.

We haven't touched the world outside the computer yet.  But we are not done.  Once we have the knowledge bits, we need some emotions bits if we are to match the human experience.  I'm not saying that we really need that.  The computer conscious could be a Spock-on-steroids who has no emotions, but that wouldn't be fun, would it?

Emotions need qualitative judgements and reactions to those qualitative judgements.  In a human being, if we are worried, our performance is impaired.  If we were to match this in a machine, worry would spawn threads that would impair performance based on whether the machine was bummed out or not.  But how would a machine express happiness?  Would it be by innocuous things like playing mp3s when it should be doing floating point co-processing?

My Australian friend said that we would never develop Artificial Consciousness.  Moi, I say that what I have brought up are mere details.  There will be frameworks upon frameworks that will give the computer a good idea of what is going on, how the computer should react, and that there is a whole other world outside the binary bits and bytes of the doped Gallium Arsenide matrices and PNP and NPN junctions in its transistor neurons. It's a scary thought when a computer first comes to the realization of an outside world, but damn, it would be exciting.   More to come.

E-ZPASS Malware Spam

The malware and virus injection spammers are getting more creative, and it has got me to wondering that this isn't originating from Russia.  Only Western hackers, and virus scum know about E-ZPass which is the transponder system on interstate highways in North America.  Some of the E-ZPass setups photograph your licence plate and send you a bill in the mail.  These guy want you to download your invoice.  Yeah, right.  Here is the email in its comic entiretyl

E-ZPass Collection Agency <>

Dear customer,

You have not paid for driving on a toll road. This invoice is sent repeatedly,
please service your debt in the shortest possible time.

The invoice can be downloaded here.

There are a couple of ironic and funny things about this piece of malicious spam.  First, it is sent from a hacked inactive site of a Catholic girls school run by nuns in Dombivli, India.  The school is call St. Therese in Dombivli.  Going to the site lights up all of my antivirus protection and Chrome won't go there as well due the virus-laden site.  But if you look at the footer of the email, they have an entry call Phishing Policy, and that is exactly what they are doing -- phishing for your private information.

You would think that since this is a Window virus, Microsoft would have solved the virus problem long ago, but they don't have the will to, or Windows is such a piece of crap that it cannot be done.

Microsoft Word ~ Capitalizing Each Word

I had a list of things in a Word document and I wanted to capitalize each word.  I didn't know how.  I had to look it up. Here is how you do it.

Make sure that you are in the standard Home tab.   In the Font area of choices, look for the icon markedwith “Aa”.  After making sure that your text is selected, click on this icon & choose "Capitalize Each Word", the last of the selections as shown:

  • Sentence Case
  • Lower Case
  • Capitalize Each Word

Hope this helps someone.

27 Ways To Make Your Webpages Load Faster

 27 ways to make your webpages load faster

  1. Avoid a character set in the meta tag 
  2. Avoid bad requests
  3. Avoid CSS @import 
  4. Avoid landing page redirects
  5. Combine images using CSS sprites 
  6. Defer parsing of JavaScript 
  7. Enable gzip compression 
  8. Enable Keep-Alive 
  9. Inline small CSS 
  10. Inline small JavaScript 
  11. Leverage browser caching
  12. Minify CSS 
  13. Minify HTML
  14. Minify JavaScript
  15. Minimize redirects 
  16. Minimize request size 
  17. Optimize images
  18. Optimize the order of styles and scripts 
  19. Prefer asynchronous resources
  20. Put CSS in the document head 
  21. Remove query strings from static resources 
  22. Serve resources from a consistent URL 
  23. Serve scaled images 
  24. Specify a cache validator 
  25. Specify a character set early 
  26. Specify a Vary: Accept-Encoding header
  27. Specify image dimensions 
Hope this helps.

The "I Quit" Spam Insult Spam

I tell you, the spammers and malware guys are getting more and more entertaining.  I recently got a piece of spam that had the subject:

Subject:  Can you handle $365 every 100 minutes?

When I looked at the body of the message, it said this:


I'm done taking crap from you. I've worked myself to the bone for 8 years now and you don't appreciate a thing I do for you.
You're arrogant, egotistical and you have no loyalty whatsoever. Every time I EVER did anything right, you always took credit for it…and I let you…because I was scared of losing my job.
A true leader takes the blame when things go wrong, and praises the people who did the job when things go right…YOU do it the opposite.

I found someone who takes me seriously: blah blah lots of parameters

I don't need you anymore.


See you tonight.

1342 ar allen RD
Beaverton, OR 97008,
Canada, 23r4rd street

It's hilarious.  The URL domain for the link is  I didn't bother checking it out, because of lot of sites that inject malware or viruses are legitimate sites.  They have been hacked with a weak password and the hackers add a new page that injects malware or viruses.  The new page is not seen unless you log in as admin and do a directory listing of all of the directories and look for a file that doesn't belong.

Note the address.  They can't decide where they are in Canada or the US with fake street names that don't even make sense.

I hope that nobody falls for this, but I bet that a fair amount are tempted just by sheer curiosity.

The Twegomaniac ~ The Twitter Follower/Unfollower Egomaniacs

My goodness, call me James Audubon and fly me to the moon.  I have discovered a new species of bird ~ the Twegomaniac.  This was a rare big that is becoming quite common.

The bird is part of fauna that exists in the Twitterverse.  There cry is "Follow Me, Follow Me" and to get you to do that, they follow you first.  But they are like the cuckoo bird who fools its host.  Once you abide by the Twitter netiquette and follow them, they unfollow you.  It's all about them.  They are the birdbrain egomaniacs who thinks that size matters.  They were born with a small size, and to try and enlarge it, instead of having the personality and content to get followers (like Jesus, Ghandi or Clint Eastwood's Empty Chair), they have to resort to trickery.  They just don't have the right stuff.

So how do you know if you have come across a Twegomaniac in your follow list?  The biggest clue is that you have never heard of them, and they have 100,000 follows and 200,000 followers.  The other identifying feature is a big asshole.

So how do you protect yourself from these odious creatures and droppings that they leave all over your home page before they leave.  There are a plethora of tools out there to help detect unfollowers.  I have been known to use the free aspect of as a pest control method.  There are many more.  Beware of the Twegomaniac.

MySQL - Getting a List of Users by who logged in last

I have a list of users.  I collect their activity. I want to get a list of users, but I want to order it by who logged in last.  Some users haven't logged in in months and don't appear in the login table.  There are two tables.  One is a users table and the other is a loginTable.  The login time is recorded by a timestamp.  Here is one way of getting the list and ordering by the last login.

SELECT, MAX(loginTable.timeIn), loginTable.userId  FROM  loginTable
right join users on loginTable.userId =
ORDER BY MAX(loginTable.timeIn) DESC ;

The right join insures that we get all of the users, even those who don't appear in the loginTable.  This is a slow query for a large user table, but since we are collecting metrics it doesn't matter.

Hopes this help someone.

Smart Road JSON or XML Template For Messaging and Data Transmission and Receiving

With the Internet of Everything here among us, and GPS is ubiquitous in vehicles, and cars will have full internet capability, it will not be long before we have Smart Roads.  Each road will have an IP address.

I began wondering what the data package would look like for Smart Roads, and then I was struck with the axiom that the best way to predict the future is to invent it.

So without further ado, I cracked together and XML file of what the Smart Road dataset would look like.  Here is my first crack at creating a Smart Road Data Standard:

<?xml version="1.0" encoding="UTF-8"?>
<!-- Smart Road Markup Language version 1.0-->

<!-- Main Element.  It consists of a header, data and trailer elements. -->

      <!-- Header Element.  -->

      <ref_number></ref_number> <!-- This could be a database Primary Key Identifier.  -->
      <ip6_address></ip6_address> <!-- Each Smart Road will have their own IP address.  -->
      <classification></classification> <!-- There could be subclassifications like carriageways etc.  -->
      <options></options> <!-- This could be a control element for suppressing some header info on subsequent exchanges.  -->


  <data> <!-- Message Section.  -->
     <general_data> <!-- This section contains general data sent to the vehicles  -->
                 <ack_flag> </ack_flag>
      <location_specific_data><!-- This section contains location-specific sent to the vehicles  -->
               <user_specific><!-- It is anticipated that if a vehicle's onboard messaging is not working, one can billboard messages -->
               <!-- This can also be used to send violation notices and service-related messages to the vehicle -->
                     <ack_flag> </ack_flag>

  <!-- Big Data meta data on usage stats etc.-->
    <protocols_supported> </protocols_supported><!-- Various devices will have their own native Smart Road protocols -->

This is just a first iteration.  It needs to be tested and validated in real time.  It is anticipated that the vehicle will send its GPS coordinates to the database defined by the IP address, and the location specific data will be returned.

Java - How to calculate and find out how many seconds to an event in the future

Java lesson.  Let's suppose that you have an event happening in the future.  The date and time of that event is stored in a database and you want to calculate how many seconds there are to that event. Once you get seconds, you can divide by 60 to get minutes, and again by 60 to get hours and then by 24 to get days etc.

Here is a code snippet that lets you do that:

Date now = new Date();
Timestamp eventStart = rs.getTimestamp("eventStart");
GregorianCalendar gc = new GregorianCalendar();
Date endInterval = gc.getTime();
long difference = Math.abs(endInterval.getTime() - now.getTime());
long secondsToEvent = difference / (1000);

Date is java.util.Date. Timestamp is java.sql.Timestamp.

Hope this helps someone.

How to find and show line numbers in Wordpad

I had an exception, an error that was being thrown from a compiled .jsp file.  Strangely, the console would not give me the line number of the uncompiled jsp, but rather it gave me the line that was causing the exception in the compiled jsp file that was in the work directory.

I didn't feel like opening the file in text editor in Eclipse to locate the line, so I opened the jsp_java file with Wordpad.  The trouble is that Wordpad doesn't show line numbers.  I found the line numbers in Wordpad with this hack:

1) Select the entire text with Select All (CRTL +A will do it).

2) Expand the bullets menu item and click on the numbered list eg 1. 2. 3. etc

3) Every single new line will now be numbered and you scroll down to the offending line.

Hope this helps someone.

Code Snippet for Creating Secure Token in Java

Tokens can be used for many things.  Among them are Single Sign On, or saving an identification number without exposing it to the web.  Here is a quick code snippet to create a 16 character token

First you need an import:


here is the code snippet:

String token1 = null;
SecureRandom secureRandom = new SecureRandom();
String secString = new BigInteger(130, secureRandom).toString(32);
token1 = secString.substring(0, 15);  //creates a 16 character token

Alter the substring end value to change the size of the token.

Hope this helps someone.

Amazon Mobile Ads API


Dear Developer,
We are very excited to announce that the Amazon Mobile Ads API is now
available for iOS apps. Now you can monetize apps across platforms,
including Android, iOS and Kindle Fire tablets and Fire phones.

With the Amazon Mobile Ads API, you have the opportunity to earn
more and you are paid on ad impressions served instead of clicks on
 ads. The Amazon Mobile Ads API is easy to integrate and delivers
highly-relevant banners, rich media and higher paying interstitials
from Amazon and brand advertisers ensuring great experience for
 your users.

With this release, we are also expanding ad serving to users in Italy
and Spain across both Android and iOS. Starting today, the Amazon
Mobile Ad Network is serving ads to Android and iOS users in the US,
 the UK, Germany, France, Spain and Italy.

Developers currently using the Amazon Mobile Ads API in their iOS
 apps share their experience*:

“Amazon Mobile Ads API is consistently delivering 20% higher eCPM
 on both Android and iOS platforms compared to other ad networks.
In addition to increased earnings, the high-quality tailored ads provides
 an engaging experience for our users achieving 25% higher CTR”
                               - Tim Oswald, Co-Founder & CEO of Harpan LLC.

“Amazon Mobile Ad Network is a top ad network for our iOS and Android
apps. Our eCPM with Amazon Mobile Ad Network are higher across the board.”
                              - Todd Moore, Founder & CEO of TMSOFT LLC.
* Your results may vary.

For a limited time, when you send the first ad request from a qualified app,
 you will receive a guaranteed $6 CPM on interstitial ads (up to a maximum
of 1 million impressions per app per month). Qualifying apps now include
 iOS apps. Please find more details here.
    To get started with the Amazon Mobile Ads API check out our Developer
    Portal here as well as our blog with tips on ad mediation and tracking ad 
    events inside your apps.

    Best Regards,
    Your Amazon Mobile Ad Network team

    Diffusion Without Photoshop Filters

    (click on picture for panorama view)

    I photographed this bucolic scene in the country, and I knew that because of the dramatic clouds and fields that I was going to make it look like a painting.  My normal way of doing this, as an uber geek, is to feed it into Photoshop, and use an array of software filters.

    However, it was raining that day, and I had my camera in a ziplock bag.  I decided to shoot through the ziplock bag for a diffusion screen.  It turned out fantastic.  The result is above.  It is a great way to get diffusion without Photoshop software image manipulation filters.

    A Java class to escape illegal characters

    Here is a Java Class to escape illegal characters with a static method:

    package com.mypackage;

    import java.util.HashMap;
    import java.util.regex.Pattern;

     * The Class StringHelper.
    public class StringHelper {

    * Escapes characters that have special meaning to regular expressions.

    * @param s
    *            String to be escaped
    * @return escaped String
      public static String fixString(String s){

        int length = s.length();
        int newLength = length;
        // first check for characters that might
        // be dangerous and calculate a length
        // of the string that has escapes.
        for (int i=0; i<length; i++){
          char c = s.charAt(i);
          if (!((c>='0' && c<='9') || (c>='A' && c<='Z') || (c>='a' && c<='z'))){
            newLength += 1;
        if (length == newLength){
          // nothing to escape in the string
          return s;
        StringBuffer sb = new StringBuffer(newLength);
        for (int i=0; i<length; i++){
          char c = s.charAt(i);
          if (!((c>='0' && c<='9') || (c>='A' && c<='Z') || (c>='a' && c<='z'))){
        return sb.toString();

    Hope this helps someone.

    Upcoming CloudKit Data Wipe

    This email came from Apple:

    Dear Developer,

    We will be performing a server-side data wipe on all CloudKit public and private databases for iOS 8 beta and OS X Yosemite Developer Preview users on July 7, 2014.  The following iCloud features will be affected: iCloud Drive, iCloud Photo Library, and Mail Drop. 

    Photos and videos stored in iCloud Photo Library will remain on their original iOS 8 devices and will upload to iCloud Photo Library again automatically. iCloud Drive can be re-enabled from Set up Assistant after upgrade. If you choose to store your documents in iCloud Drive, your Documents & Data will automatically be copied to iCloud Drive. iCloud Drive will not update across earlier seeds or operating systems. Attachments sent through Mail Drop will expire and need to be resent after you upgrade.

    If you have any questions, visit the Apple Developer Forums.

    Best regards, 
    Apple Developer Technical Support

    Getting SSO to work

    We recently decided to use for our help desk, knowledge base etc.  Since we want our knowledge base to be private for clients only, Freshdesk requires a login.  We don't want to make our clients log in twice, so Freshdesk has this SSO or Single Sign On token system.

    We use Java, and there was a java servlet showing how to sign on with a token.  However the online java in a repository didn't quite work.

    Here is a working Java servlet for the Freshdesk SSO token system:

    import java.math.BigInteger;

    import javax.servlet.ServletException;
    import javax.servlet.http.HttpServlet;
    import javax.servlet.http.HttpServletRequest;
    import javax.servlet.http.HttpServletResponse;

    import com.application.User; (replace with your own user bean)

    public class FreshDeskSSOServlet extends HttpServlet {

    private static final long serialVersionUID = 7027717204177362374L;
    private final String BASE_URL = "baseUrl including trailing slash" + "login/sso";
    private final String sharedSecret = "put in shared secret";


    public void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException, ServletException {

    try {
    String url = this.getSSOURL();
    } catch (Exception e) {

    private static String getHash(String text) throws Exception {
        MessageDigest m=MessageDigest.getInstance("MD5");
        return ""+new BigInteger(1,m.digest()).toString(16);

    private String getSSOURL() {

    String hash = null;
    String url = null;
    User user; //Get the user details using your current authentication system
    String name = "First & Last Name";// Full name of the user
    String email = "";// Email of the user
    try {
    hash = getHash(name + email + sharedSecret);
    } catch (Exception e1) {

    try {
    name = URLEncoder.encode(name);
    email = URLEncoder.encode(email);
    url = BASE_URL + "?name="+name+"&email="+email+"&hash=" + hash; 

    }catch (Exception e) {
    //Handle appropriate code
    System.out.println("There is an exception while constructing the URL");

    return url;


    Hope this helps.