So, the immediate costs to use are that you have to research a bunch of parsers to see which fit your needs and are 'good'. Then you have to write the code to use them, which depending on their interface may itself add a significant number of lines of code with potential for their own bugs (given a simple enough example, the integration code can be larger than the code you would have written). Then you have to include not just the library, but all its dependencies in your codebase. You're then committed to keep track of the library and all its dependencies, and updating them if and when dangerous bugs are discovered. This means that your code now has a reason to change or rot (API changes from upstream) that has nothing to do with you or any feature or bug you've written. If your requirement is later found out to be slightly unusual and not exactly covered by the library, you're in for a world of pain working around that too (e.g. in the example given, this probably isn't a valid xml file because it doesn't have a single surrounding tag and there may be other nonstandard issues in the input).
The amazing thing is that this actually is worth it in many cases, but people often underestimate the cost.
Sure, the poster you're replying to isn't a special flower that can write a better parser for whatever format, but he's saying that in this case he doesn't need one. A number of other commenters have pointed out that requirements change and the generic parser will be a better choice than evolving your homegrown simple code into a poorly thought out monster. This is true, but it's also premature optimisation.
The right time to pull in those libraries is on the first non trivial change, not at the beginning.
>So, the immediate costs to use are that you have to research a bunch of parsers to see which fit your needs and are 'good'.
Yep. This is known as an investment. It keeps paying dividends every time you use it.
>Then you have to write the code to use them, which depending on their interface may itself add a significant number of lines of code with potential for their own bugs (given a simple enough example, the integration code can be larger than the code you would have written).
Only if you're a bad programmer, and/or you used a bad library.
>Then you have to include not just the library, but all its dependencies in your codebase.
For me this means adding one line to a file called "requirements.txt". E.g. python-cjson==1.1.0. The setup scripts take care of the rest. Trivial.
>You're then committed to keep track of the library and all its dependencies, and updating them if and when dangerous bugs are discovered.
Yep. Once a month or so I upgrade all of my libraries, run the tests and see what breaks. Mostly nothing breaks and the amount of thinking I have to do to take care of this problem is usually zero.
>This means that your code now has a reason to change or rot (API changes from upstream) that has nothing to do with you or any feature or bug you've written.
While true, stuff like cjson just doesn't change its APIs. In practice, while this is a problem for some libraries, it's one that is relatively minor.
The reverse is more usually true: upgrade everything and you get bugfixes for free. Good library designers do not change their APIs often but they do fix bugs frequently.
> If your requirement is later found out to be slightly unusual and not exactly covered by the library, you're in for a world of pain working around that too (e.g. in the example given, this probably isn't a valid xml file because it doesn't have a single surrounding tag).
Hence "choose a good library". For xml that means choose one that can deal with invalid XML because invalid XML is a problem you're just going to have to deal with eventually if you use XML. I use libxml2. It's good. It's fast and it handles invalid XML well. I recommend it.
Better yet, don't use XML coz it's crap (not always avoidable, I know).
>The amazing thing is that this actually is worth it in many cases, but people often underestimate the cost.
The cost is higher if you choose poor libraries. Choosing a good library is more art than science, but if you do it right you will drop your costs like a rock.
It also helps to pick a big ecosystem + language that has good libraries.
>Sure, the poster you're replying to isn't a special flower that can write a better parser for whatever format, but he's saying that in this case he doesn't need one. A number of other commenters have pointed out that requirements change and the generic parser will be a better choice than evolving your homegrown simple code into a poorly thought out monster. This is true, but it's also premature optimisation.
This is a misuse of the term premature optimization. You can prematurely optimize for speed. You cannot prematurely optimize to eliminate not invented here syndrome, code duplication or tight coupling.
>The right time to pull in those libraries is on the first non trivial change, not at the beginning.
No, the right time is at the beginning. If you need to parse JSON use a json parsing library. If you need to parse XML, use an XML parsing library.
If you're worried google what people have to say about it. Do they say that it sucks? It's glacially slow under the wrong conditions?
If you write loosely coupled and cohesive code as well, you can vastly reduce the costs of choosing a bad library later on.
So, the immediate costs to use are that you have to research a bunch of parsers to see which fit your needs and are 'good'. Then you have to write the code to use them, which depending on their interface may itself add a significant number of lines of code with potential for their own bugs (given a simple enough example, the integration code can be larger than the code you would have written). Then you have to include not just the library, but all its dependencies in your codebase. You're then committed to keep track of the library and all its dependencies, and updating them if and when dangerous bugs are discovered. This means that your code now has a reason to change or rot (API changes from upstream) that has nothing to do with you or any feature or bug you've written. If your requirement is later found out to be slightly unusual and not exactly covered by the library, you're in for a world of pain working around that too (e.g. in the example given, this probably isn't a valid xml file because it doesn't have a single surrounding tag and there may be other nonstandard issues in the input).
The amazing thing is that this actually is worth it in many cases, but people often underestimate the cost.
Sure, the poster you're replying to isn't a special flower that can write a better parser for whatever format, but he's saying that in this case he doesn't need one. A number of other commenters have pointed out that requirements change and the generic parser will be a better choice than evolving your homegrown simple code into a poorly thought out monster. This is true, but it's also premature optimisation.
The right time to pull in those libraries is on the first non trivial change, not at the beginning.