Quick nav:
Oracle’s LMS audit of the City of Denver
Tactics used by Oracle LMS
What could Denver have done differently?
In a CBS News investigative report, Brian Maass uncovered a large Oracle audit on the city of Denver that resulted in Denver paying Oracle almost $4m. While this type of audit might have been unknown to the innocent taxpayers of Denver, at Palisade we’ve seen this move by Oracle over and over again.
In the past, we’ve talked about these ABC Audits – Audit, Bargain, Cloud. But now, completely through documents available in the public domain, we can show you exactly how Oracle does this.
Step 1 – the audit
The City of Denver received an audit letter from Oracle on July 27, 2016. Below is an email from Oracle to the City of Denver.
Despite the client’s contract providing an extended notice period, Oracle insisted on starting the audit within 3 days.
Denver asked that all audit activity be done over email, because, according to one Denver employee: “Colorado has an Open Records Act, and we value transparency in government. We want to make sure that if any member of the public wishes to scrutinize the actions of either party, that we have a complete and accurate trail of correspondence.” (August 30 email.)
Despite this, Oracle insisted on a face-to-face meeting. Denver sent another request to the Oracle LMS Audit team stating, “Concerning the conference call and face-to-face meeting that you [LMS] have requested, we reiterate our earlier request that all audit-related communications continue in writing.” (September 15 email.) To which Oracle LMS responded, “We are now officially demanding a face-to-face meeting to conduct the audit.” In addition, as an added threat, Oracle LMS brought in their legal counsel in the same email, which continues, “Oracle’s Legal Counsel, is copied on this e-mail. Please provide the contact information of City and County of Denver’s attorney should the matter need to be handled by our respective Counsels.” (October 4 email.)
There you have it; a legitimate request by a customer is met with the threat of legal escalation by Oracle’s LMS audit team.
Step 2 – the bargain
The on-site LMS audit meeting was held on Thursday, October 20, 2016.
It’s interesting that we were not able to find an interim LMS audit report or final LMS audit report in our research. What we do know is that Oracle decided that Denver was out of compliance, and Denver agreed there was a compliance problem. In fact, on November 28, a little more than a month after the LMS audit meeting demanded above, the CIO of Denver wrote to Oracle and said, “The City and County of Denver would like to move forward on “right sizing” our licensing agreement.” (November 28 email.)
Something happened between October 20, 2016 and November 28, 2016. We didn’t see any reports produced by Oracle that show the non-compliance allegations. Maybe there was a report; we just have not seen it. Why is this important? Well, at Palisade we’ve been saying for years that Oracle’s LMS audit team does not have to finish the audit to drive the deal to completion with Oracle sales.
In fact, by not finishing the audit, Oracle makes more money. LMS can put some fear into the customer and have the sales teams close a new deal, while LMS moves on to their next target.
Step 3 – the cloud
Remember, Denver was audited for their use of on-premise software. They were not audited for cloud usage. Therefore, any non-compliance findings should have resulted in Denver acquiring licenses for on-premise software. In this case, Oracle sales put two offers in front of the client: one with cloud, and one without.
The email excerpt below shows both options. While the math is not exactly what Denver signed up for, you can clearly see the message: buy Oracle cloud or we will make you pay more money. In the case below, almost $2m more if the customer doesn’t buy cloud.
Just in case you think there’s a difference between the licenses in Option 1 vs Option 2, the Oracle sales manager who sent this letter states, “Please note that the list of products included within the Unlimited License Agreements (ULA) in both options is exactly the same.” (Letter from Oracle to Denver, December 22, 2016.) The Oracle sales manager wrote in an email accompanying this letter, “As shown, including Oracle Cloud will save the City and the County of Denver just under $2m over the next five years.” (Email December 22.)
Putting the pieces together
The dollars finally spent by Denver and sent to Oracle were slightly different from the numbers above. In the end, Denver spent approximately $3.9m total. The picture below shows the exact breakout.
The cloud numbers above of $1,895,178.96 were in fact only one year of cloud subscription services. This is very important for two reasons. First, for the City of Denver, it means that if Denver actually used the Oracle cloud, then the Denver taxpayers will have to pay Oracle almost $2m a year MORE than the numbers indicated in the sales offers above. Oracle sales’ numbers were only based on a one-year cloud subscription. Any cloud usage more than a year means the on-premise-only offer would have been less money.
Secondly, and most importantly for us who follow Oracle, is that it appears Oracle was only selling a one year cloud subscription to Denver with no anticipation of Denver actually using the cloud beyond the first year. If Oracle sales thought Denver was going to use the Oracle cloud, then they would have priced it out in their option 2 offer above and the 5 year cost for Denver would not be $6.1m, it would have been approximately $14m.
Now we can put all the pieces together. Had Denver not been audited, they would not have bought the Oracle cloud subscription. If buying only the on-premise licenses was less expensive, Denver would not have bought the cloud subscription.
It is fair to conclude Denver did not buy Oracle cloud because they wanted Oracle cloud. Denver bought Oracle cloud because Oracle pushed Denver into a corner with coordinated LMS audit and sales pressure tactics.
Only time will tell if Denver actually uses the Oracle cloud they bought. The information we have in the emails and contracts we’ve reviewed showed nothing about Denver acquiring this cloud subscription for any reason other than Oracle making it less expensive than buying on-premise only. The Oracle LMS audit of the City of Denver has all the makings of the ABC Audit – Audit Bargain Cloud type.
Oracle LMS audit tactics – City of Denver
What are some of the specific tactics Oracle LMS used against Denver so you can be prepared should Oracle LMS use those tactics against you?
Because Oracle audited Denver, a public sector entity, the communications of this audit were subject to Denver open records rules, and we were able to obtain much of the back and forth. It is interesting that the written record gets sparse once Oracle identifies a compliance problem. That being said, we have identified at least three tactics in the Denver audit that Oracle LMS/sales will use in audits all the time.
Tactic 1: rush to meet
The first Oracle LMS strategy is to bypass the notification safety period in their customer’s contracts. Oracle has the right to audit their customers. There is a contractual provision that allows Oracle to perform that audit. In that provision, there is a notice period that says Oracle can audit upon 30/45/60 days’ notice to the customer. Different customers have different notice periods, so we can’t be 100% certain what Denver’s notice period was. We do know it was at least 30 days. Oracle LMS knows about this notice period. However, as you can see in the audit letter excerpt at the top of this case study, Oracle demanded a meeting with Denver within three days.
This is a classic Oracle LMS tactic. Rather than letting the customer fully prepare and get ready for the audit, Oracle wants to get in there right away and start the process. It would be very easy for Oracle LMS to simply state in their audit letter that they would like to start the audit after those 30 or 45 days. Why do you think they don’t do this? The answer is that a prepared customer is more difficult to audit and come out with a compliance finding. Oracle LMS is in there to produce revenue and make money for Oracle. That’s why the rush.
Tactic 2: legal threats
Oracle LMS is known for their aggressive audit tactics. We literally can’t print what some of our clients have said about that organization.
In the case of Denver, the Oracle LMS analyst was quick to threaten the client with a legal escalation. In fact, Oracle LMS copied the Oracle lawyer on an email where he “demanded” Denver do what Oracle wanted done.
What did Denver do that brought the hammer of legal into the situation? Well, Denver asked that the audit be done in writing, to preserve a record for public viewing, as is apparently the law in Denver. That request alone brought Oracle LMS to the legal threat. It’s a tactic we see Oracle LMS use consistently. And it’s very effective when they are auditing a customer that does not know how to respond. In this case, Oracle’s threat worked; the face-to-face (not email) process was followed, and Oracle quickly extracted $4m out of Denver. $4m for a threatening email? Not bad.
Tactic 3: hit the customer with a huge compliance number
The record is bare when it comes to exactly what Oracle found that caused the compliance issue with Denver. We do know that Oracle stated it could be in the $10m range – a huge number. What that number really does is let the Oracle LMS and sales teams put another large number on the table, in this case, over $3m in licenses and cloud fees, and claim that the customer is getting a deal. Look at how great we are being to you. We could charge you $10m, but we’re such great guys we are only going to charge you about $3m! In that context, then the audit target doesn’t feel as bad as the audit finding $3m of non-compliance and they paid $3m in non-compliance.
We see this tactic all the time, too. Oracle LMS running up the score on a client and trying to put the biggest number on the board so they can hand it off to the Oracle sales team and they can close the biggest deal. And yes, once again, it is very effective. You just have to look at the City of Denver and see how quickly (within weeks) they went from a face-to-face audit meeting to agreeing with Oracle on a deal. It’s Oracle’s version of “shock and awe”, only this time it’s inflicted on the customer by the Oracle LMS audit team and sales team working together.
What could Denver have done differently?
The short answer is – A LOT! The longer answer may surprise you. Denver had a lot more control over this audit than they were aware of.
Here’s our deconstruction of the Denver response and definition of what Denver could have done differently to completely avoid, or at the very least dramatically reduce, the Oracle LMS audit penalty.
Denver should have got help
The first thing Denver should have done was get help from the right people. Getting help makes all the difference in the world. Had Denver reached out to Palisade Compliance, you can be sure that Oracle would have adhered to the contractual notice period, as well as the “everything in email” requirements. The entire audit would have started from the client being in a position of strength and control. There are other things that would have been done differently as well.
Make sure you get help from the right team. Don’t ask an Oracle reseller, tool vendor, or Oracle itself to help you with your audit. That kind of help is a disaster waiting to happen.
Have a tight audit response plan
It appears from the materials reviewed that Denver did not have a tight audit response plan when Oracle knocked on the door. If they did have a plan, it was not effective, because Oracle LMS crashed through every objection we saw and really ran the audit according to Oracle’s standard playbook.
If Denver had a plan, they could have imposed that on the process. Oracle contracts don’t really specify how an audit will be conducted. If it’s not in the contract, then it’s up for negotiation. We didn’t read any emails where Denver told Oracle they have a software audit plan/playbook, and Oracle had to follow it. Big mistake.
When Oracle escalates, escalate higher
Oracle LMS brought their attorney into this audit after Denver requested the audit be documented in email. That legal escalation threat appeared to have worked, as Denver quickly agreed to Oracle’s “demand”. In cases like this, Denver could have gone way up the Oracle food chain and escalated to very senior executives and attorneys. If there really was an open records obligation, Denver should have pushed it, and made Oracle put everything in writing for all to see. I’ve seen Oracle’s customers escalate all the way to the CEO. Oracle LMS has no problems writing your c-level executives; you should have no problem doing the same thing with them. The key is in knowing who to escalate to, when to escalate, and what to say.
Know the answer before Oracle
Oracle’s audit process is designed for you to give them all the information, in a format you can’t decipher (script output), and then put it in their black box until they tell you what they’ve found. And Oracle will keep looking until they find something.
Anyone who is audited by Oracle, including the City of Denver, should know their compliance position before Oracle knows it.
That’s only common sense. You want to put your company in the best possible situation. In fact, you can even tell Oracle what the results are and have Oracle try and argue with your facts. Don’t let Oracle create its own set of facts.
Follow the contract
If it’s not in the contract, it doesn’t exist. That’s a good rule to follow with an Oracle audit. We could be wrong, but given the size of the numbers being thrown around in the Denver audit, we can only assume that the issues were around virtualization. Denver’s Oracle contract probably didn’t say anything around virtualization. So why did Oracle get to pick and choose this policy to insert into this audit? The answer is simple: because Denver let them do it. Had Denver forced Oracle to stick to the contract, this issue would have been resolved in a totally different way. This is where there is an art to an Oracle audit response. You need to know when to force the contract, and when to let it go.
Oracle LMS Denver audit wrap-up
Using information available in the public domain and exposing what Oracle LMS and sales do in an audit situation should be both comforting and alarming for every Oracle customer. On the one hand, there are things Oracle customers can do to better manage an audit and severely reduce or eliminate any non-compliance finding.
On the other hand, Oracle has more than 400 people in their audit department who do nothing but target and audit their customers. Day in and day out. In addition to Oracle, there is a slew of Oracle resellers, tool vendors, jack-of-all-trade firms, and individuals who claim they can help you in an Oracle LMS audit. It’s “buyer beware” when using one of those firms.
Palisade Compliance can assist any Oracle customer at any stage of an LMS audit (from beginning to end).
