> No one reads them. It's just some stupid long-tailed distribution where three of these absolutely useless fucking things gets read a few times a day (by someone that probably doesn't do anything with the information) and the rest of them don't get read at all.
My first job out of college was on a cargo logistics team where we were tracking multiple pieces of very, very expensive cargo that had to be packed in a certain way while in transit to their destination (which rhymes with "Blinternational Blace Blation"). The job wasn't terribly difficult; our node in the planning chain was a database with about 80,000 records with maybe 110 properties per record, and most were managed by the system. We were actively tracking 5-8000 at any given time.
There were tons of issues in the overall cargo logistics process, because at each "node" in the process, no one (hardware owners, hardware coordinators, packing technicians) really cared about data sanitation. This was my first job, so as I researched the root causes of problems I'd find core design deficiencies, like "Desired Shipment" being a free-text entry field instead of a pick list, or "[Status] = Approved" meaning one thing to one group and a different thing to a different group.
Eventually, I'd grown so competent that I had a list of about 45 checks to do on any individual record, and I was using Power Query to create automatic reports on how many records per shipment were corrupted, incomplete, or otherwise likely to cause issues. I'd get hauled in to various large working forums to explain why processing was taking too long on some $10M microscope and I'd pull out my dashboard and reports to show that the hardware owner hadn't entered in dimensions or mass yet, and that there were 17 more records that had the same deficiency, yadda yadda yadda.
No one ever really did anything with all the work I put together, and every time we suggested something some division would have an elaborate explanation for why they wouldn't possibly sanitize their data or provide it earlier in the process. Last I heard they're paying millions of dollars per year to develop an end-to-end platform to do exactly the same thing I was doing (query multiple databases, compare records, spit out lists of incomplete records). I doubt that expensive database-reading tool will fix the problem either, because it's not even just that "no one reads the reports"; in addition to not reading the reports, they don't know what to do with the insight.
My first job out of college was on a cargo logistics team where we were tracking multiple pieces of very, very expensive cargo that had to be packed in a certain way while in transit to their destination (which rhymes with "Blinternational Blace Blation"). The job wasn't terribly difficult; our node in the planning chain was a database with about 80,000 records with maybe 110 properties per record, and most were managed by the system. We were actively tracking 5-8000 at any given time.
There were tons of issues in the overall cargo logistics process, because at each "node" in the process, no one (hardware owners, hardware coordinators, packing technicians) really cared about data sanitation. This was my first job, so as I researched the root causes of problems I'd find core design deficiencies, like "Desired Shipment" being a free-text entry field instead of a pick list, or "[Status] = Approved" meaning one thing to one group and a different thing to a different group.
Eventually, I'd grown so competent that I had a list of about 45 checks to do on any individual record, and I was using Power Query to create automatic reports on how many records per shipment were corrupted, incomplete, or otherwise likely to cause issues. I'd get hauled in to various large working forums to explain why processing was taking too long on some $10M microscope and I'd pull out my dashboard and reports to show that the hardware owner hadn't entered in dimensions or mass yet, and that there were 17 more records that had the same deficiency, yadda yadda yadda.
No one ever really did anything with all the work I put together, and every time we suggested something some division would have an elaborate explanation for why they wouldn't possibly sanitize their data or provide it earlier in the process. Last I heard they're paying millions of dollars per year to develop an end-to-end platform to do exactly the same thing I was doing (query multiple databases, compare records, spit out lists of incomplete records). I doubt that expensive database-reading tool will fix the problem either, because it's not even just that "no one reads the reports"; in addition to not reading the reports, they don't know what to do with the insight.