This is a memo we sent to the appraisal industry today.
To: All appraisers
From: Dave Biggers, Chairman, a la mode, inc.
RE: Collateral Underwriter – What to Do
Unless you’ve been hiding under a rock, you’ve heard lots of stories about how the world will soon be turned upside down by Fannie Mae’s Collateral Underwriter appraisal review tool. You’ve been told that weeks of chaos and endless stipulations will be added to every appraisal report. You’ve heard that you’re already behind in this “arms race” of brains versus statistics and computers. And you’ve of course been told that you need to spend money, yet again, to buy more software (even though it won’t generate any additional income).
Our recommendation, however, is simple and contrarian: Don’t fall for the fear mongering. Save your money.
Why? Any changes you might need to make in your appraisal habits will probably be minor; many of you won’t need to change a thing. You’ve been getting your reports reviewed for years by tools and checklists that are a lot more intrusive, and usually based on horrible public records and MLS data. Collateral Underwriter uses better data, generates fewer items to check, and isn’t even a new development at all – Fannie has been running it for years to evaluate your reports. The big difference is that now they’re allowing lenders to use it too.
Remember too, if you’re a TOTAL or Aurora user, the extra tools we’ll be adding to help you adapt to CU’s review messages will be absolutely free, as part of your Silver, Gold, Platinum, or Elite membership. (Just like we did with UAD, and just like we’ve done for decades.)
Over those same decades, we’ve built a track record as staunch appraiser advocates, raising the alarm loudly on issues that we perceived as threats to you. (After all, if you make less money, or you’re at risk, then the same is true for us.) If we thought this fell into that category, we’d absolutely tell you. But we don’t see compelling evidence that there’s a coming Armageddon, and we’ve seen thousands of reports go through Collateral Underwriter (let’s just call it “CU” for short). We could always be wrong, but after looking at it in great detail, we don’t think we are.
On that note of looking at things in detail, I’ve been shocked at the number of people spending hours upon hours online, fretting over CU, without spending an equal amount of time researching what CU actually does. So if you haven’t already, please visit the information page on Fannie Mae’s website, at https://www.fanniemae.com/singlefamily/collateral-underwriter. Once you’re there, be sure to watch the two videos under the section titled “OnDemand eLearning Course”. Those videos might seem a bit ominous at first, but after we look at it step-by-step, and add some common sense and clarity, CU should be less frightening.
Let’s start with the CU “messages” or “flags” that get kicked back to the lender when an appraisal is submitted to UCDP. While there are technically over a hundred messages, there are actually only 95 that affect you directly. And you’re highly unlikely to ever encounter most of them.
You can click here for a printable PDF of those 95 messages. You’ll see that we’ve arranged them into four logical categories. And in the table below, we’ve also summarized them with general descriptions and our take on the issues surrounding each category. We think any objective analysis of CU’s messages would generally draw similar conclusions to ours, but we’re always open to debate.
The first thing that stands out is that most of them are under your direct control – simple oversights, honest mistakes, guideline lapses, and so on. In fact, 52 of the 95 flags – more than half – are in the Appraiser Consistency and Errors categories, covering basic issues that your clients have been checking your reports against for years. Those are easy to avoid and equally easy to fix, and usually can be caught by existing appraisal software error-checking tools – without appraisers getting mini-PhDs in statistics.
| Category | Count | Description and Notes |
|---|---|---|
| Appraiser Consistency | 13 | These are flagged when an appraiser is inconsistent with data they reported on prior appraisals regarding the same subject or comp. The most common mistakes of course will be View, Condition, and Quality, which should never be changing from report to report for any reason. Avoiding these flags is absolutely under the appraiser’s control, by re-using prior reporting of the comp or subject data religiously. These only apply to UAD fields and other “pure math” comps grid line items. |
| Errors | 39 | The largest category by far, these are logic or math errors such as adjustment direction (positive even though the comp feature is superior, etc.), contradiction with clear guidelines (new construction with “C3”), reconciled value above the range of all comps, and so on. These are clearly “self-inflicted” when they happen. |
| Adjustments | 14 | These relate to CU-calculated “model adjustments” and other guidance, such as comp selection. Adjustments for GLA, View, Site, etc. are critical. The key issue here are the words “materially different” and “versus peer and model” in the messages. In other words, you need to be way off the logical path. Though these aren’t self-inflicted or completely under the control of the appraiser, they are usually “alertable” beforehand. Using very quick non-statistical checks and common sense guidelines, steps can be proactively taken to research these items better, or document the logic more clearly. These apply to UAD fields and other “pure math” comps grid line items. |
| Peer Consistency | 29 | Similar to the “Appraiser Consistency” category, this second largest group of flags relates to material differences between the data reported by the appraiser and his or her peers regarding the same property. Entries for condition, quality, location, view, bedrooms, etc., which aren’t similar to the data from peers on the same property, will trigger a warning. Sharing data with other appraisers can help, as can sticking closely to the fairly objective criteria for assigning ratings and defining rooms. These aren’t 100% avoidable without having all the UAD data from all appraisers on hand at all times, but caution (such as “don’t overthink it”) and common sense will keep most appraisers from getting flagged. |
That leaves the two potentially “scary” categories – Peer Consistency, and Adjustments. You don’t have Peer Consistency data that you can check against (yet), and you also don’t have the same data that gets used to generate the adjustment ranges. But even those two aren’t so frightening once you look at them closely. There seem to be fairly generous thresholds before warnings are triggered, and so simple common sense steps can be taken to avoid problems.
On Peer Consistency, if you stick to the published UAD guidelines on assessing condition, quality, and so on, and faithfully use common data sources, you’ll reach similar conclusions as other appraisers. You don’t need to reach identical conclusions as all of your peers; you just don’t want to be way out on the fringe. Even if you are, you might be right, so just answer any questions you get. And as always, if you know that something will be “odd” compared to others – maybe there’s been new data that they didn’t have – then explain it in advance. Again, you’ve done this sort of thing a thousand times over the years, and I’m not telling you anything you don’t already know.
On the Adjustments message category, it’s also easier once you dissect it rationally. Stay away from arbitrarily low “rules of thumb” on things like GLA, condition, and lot size, for example. You don’t need regression analysis to tell you that adjusting a $200/sf set of comps by just $20/sf isn’t market-correct in most cases. Just use common sense.
If people will pay $200 for the average square foot in this hypothetical example, they’re unlikely to magically drop to a tiny percentage of that per square foot for similar comps with reasonably more or less feet. Let me repeat, because I can’t emphasize this enough: If there’s any bad habit that you should break, it’s using arbitrarily low GLA adjustment factors.
Historically, appraisers have been trained to under-adjust on things like GLA, site, and condition, which is one of the reasons that Fannie removed the 15%/25% rules. Even after lifting those restrictions, we’re betting CU will still be more likely to encounter under-adjustments than over-adjustments. Some of that will be client driven, because some lenders and AMCs will dogmatically stick to that rule even though Fannie removed it. But you’re better off defending why you broke the client’s 15%/25% rule than you are defending why Fannie says you’re under-adjusting on significant line items.
I’m not trying to be completely comprehensive here, so there are obviously things I haven’t covered. But the point is that there are simple first steps that any appraiser and any software can take – more comprehensive reviews, logic checks on adjustment scales, property consistency checks, etc. – and we’ll be rolling those tools out for TOTAL and Aurora users over the coming months. As I said above, that’s at no charge as part of your membership with us.
You may be wondering if regression tools will be in there. Yes, we’ll give you that too. (And there are several different integrated regression tools already in the TOTAL Store for both TOTAL and Aurora). But I have to toss in a few words of caution: Regression is being sold as magic snake oil too frequently. You’re often being pitched a cure that doesn’t work, for a disease you don’t have, and that snake oil itself can indeed kill you if you don’t know how to handle it. (I see blatantly erroneous use of statistical analysis all the time that would never survive cross-examination.)
The worst part about regression is that the data you have access to (MLS and public records) isn’t very good for regression analysis in the first place. The phrase “garbage in, garbage out” comes to mind. Consider the Adjustments category of messages in the table above for a moment: The CU model uses the UAD fields for its calculations. Other than GLA, that data isn’t in UAD format in any MLS or public records file that you’re ever going to import into a regression tool. Whereas Fannie can extract adjustments for condition, quality, view, and so on because they have tens of millions of UAD property records with objective data like C1, Q2, Beneficial, etc., you’re going to have a hard time extracting meaningful adjustments from a handful of MLS records with crappy data like “Cute fixer-upper with a darling view!”.
If MLS or public records data regression could generate accurate adjustments on the important line items on the comps grid, Fannie and Freddie wouldn’t have needed to create the UAD-restricted fields in the comps grids in the first place. They know that it takes appraiser-created UAD data to generate a valid comps adjustment estimator. And even then, it’s still going to be imperfect, so they look for “material differences” from their model’s conclusions, and from your peers, when generating flags.
There are a dozen other reasons to be skeptical of MLS and public records regression (obviously, Fannie clearly is). Until you have better data, focus on core concepts. Measure the market as well as you can, using all the techniques at your disposal. Think like the buyers, because it’s their behavior that you’re measuring. Go back to basics, like principles of substitution and diminishing marginal utility. Talk to brokers, agents, and builders. Talk to your peers.
Now, if you’re looking at that Peer Consistency category and wondering why Fannie doesn’t just give you the peer data, I can tell you that we’ve asked them the same thing. We’ve had some interesting discussions on the topic, and while they’re sympathetic to appraiser concerns, they believe they have bigger fish to fry for now in just getting lenders trained on CU overall. I don’t necessarily agree with that prioritization, but lenders are directly connected to Fannie and it’s easier to put them first. (If it makes you feel any better, AMCs aren’t direct connections to Fannie either, and they’re low priority too. They don’t have direct CU access – only lenders do – and they aren’t getting CU training either. They have to get their information from lenders.) So, the CU interface is designed for lenders’ use exclusively, and the initial training and rollout is therefore focused on them.
That training and rollout is where I do see some potential problems on the horizon. CU messages will probably start showing up on lender screens before many of them have been sufficiently trained on what they actually mean. CU’s fairly vague messages could turn into “fix this or else” demands in the hands of a poorly trained or unqualified reviewer (and nobody’s ever seen one of those, right?).
But even with that risk, we should avoid overreacting. That’s because, generally speaking, data-driven modeling systems like CU wind up looking at the things that sit more than one standard deviation outside the norm. They’re not hyper-sensitive like rigid rules-based systems. So, about two thirds of all appraisals (one standard deviation each way) are likely to have low risk scores. And on average, most will generate less than a handful of flags. And by handful, I mean that you’re likely to be able to count them on one set of fingers, if you’re proactively catching the issues you have control over.
In the short term, we recommend keeping an open mind – in terms of CU itself and the clients using it. If a reviewer or underwriter asks you to consider a comp that CU suggests, or to explain something that CU flagged, just take a deep breath first. It might be a good catch (they do have the best data after all – for now). Even if not, it isn’t worth angering a good client over early “learning curve” issues. Implementation of CU is likely to stretch all the way through the third quarter of this year (or later) in my opinion, because lenders move on a glacially slow time scale. Even HVCC and UAD deadlines were missed by many of them with many months or years of notice, so I doubt this will be different. Patience will be a virtue here.
That being said, I do think some appraisals will get held up, rightly and wrongly. There’s no question that a small percentage of appraisals are, frankly, just not very well done. They’ll get flagged – and correctly so. Others will require additional effort before they sail through CU, due to oversights or logical flaws, or possibly just quirks in the property itself, which are no fault of the appraiser. It will not be perfect. There will be speed bumps. And there will be speeders who get caught. But the bottom line is that January 26th is probably not the end of the world.
You’ll see more from us over the next few weeks and months on this issue and on a variety of other very significant topics (new stuff that you’ll like a lot). Take this quiet time of year and digest them please, along with getting up to speed on CU.
I’ll wrap up by wishing you a happy new year, in spite of all the recent fear mongering, and by thanking you for supporting us with your business. I’m confident that in 2015 you’ll see even more reasons for that loyalty, especially in terms of us continuing in our long-standing role as the independent appraiser’s staunchest advocate, and in being the most innovative, reliable, and accessible technology provider. We look forward to working with – and for – all of you this year.
Sincerely,
Dave Biggers
Chairman
a la mode, inc.
PS – I dropped some hints above that you could have access to the “Peer Consistency” data eventually, so I should probably explain more about it.
I’ve always said that I think it’s insane that everyone has access to appraiser-provided data EXCEPT the appraisers who generated the appraisals in the first place. That needs to be fixed. So, we’re creating a system enabling you to seamlessly share comps among yourselves. You’ll have the same data that’s being used to judge you.
We’re confident it will be effective: a la mode users alone generate enough appraisals per day to cover the entire market with high quality subject and comp data. With just the existing TOTAL and Aurora users joining in, tens of millions of comps will be available, exactly like in Fannie’s system – in high-quality UAD format.
How do we know that? It’s not by data mining. We’re basing these calculations on our street map statistics. When you ask for a street map in TOTAL or Aurora, our web servers scrub the addresses for you against the USPS database, then geocode all of the properties you’re mapping, and then pass that map back to your desktop. The address and type of property (subject or comp) is all we know – we don’t have your comps data, trust me. But those address lookup stats are enough to tell us that the average subject is re-used approximately six times in other appraisers’ reports over the next year. That’s consistent with what Fannie says as well. And, we know from Fannie’s statements on the CU website that the volume of addresses flowing through our map servers is about the same as what they see. In other words, when a la mode appraisers voluntarily share their data en masse with each other, they’ll basically have the same data that Fannie has access to. Instantly.
Let me be clear however: Any database we build for sharing comps will not be a database for us to use for our own purposes. The data is yours. It isn’t ours. We don’t want to sell it, use it, move it, share it, or do anything else with it other than manage it for you. (We’ll also put in place full controls to prevent others from misappropriating it.)
Building that shared appraiser-only database is easy from a technical perspective, but it’s the political part that creates risk for us. Many years ago, we built our first CompsXChange product, allowing appraisers to create “buddy lists” of others with whom they’d like to share their subjects. But there was too much suspicion among appraisers – suspicion of each other, and of us – for it to get critical traction. We finally killed the product because of complaints from appraisers who hadn’t even used it. It was a massive opportunity for appraisers to take control of their own data, and that opportunity was lost.
With the stakes so much higher today, and with Fannie and lenders already using your appraisal peer data to judge you, we’ve decided to release CompsXChange again, as part of our Comps Dashboard project. (This isn’t a new concept of course. Local appraiser-run comps sharing services have existed for decades.) It’ll be fully optional, so you’ll be able to choose whether or not to participate. I can’t imagine why anyone wouldn’t – certainly not today.
Help us spread the word about how your data can finally be in your own hands, not everyone else’s. It’s critical. You’ll see more about it from us in the coming weeks, so stay tuned.
