A heuristic guide...If you have more data than knowledge about the domain, and you want to do classification/prediction, then NNs are a good.
PGMs are good if you have knowledge that is important to encode; you need modularity i.e. you don't want to embed priors in the model; you have strong causal relationships e.g. diagnostic models with 'explaining away'; you want to integrate value of information e.g. what test should I do next to resolve uncertainty.
You can mix ML with knowledge in PGMs. The downside is the computational complexity of inference which is NP-hard for both exact and approximate inference, although you can identify the complexity of the model at design time and make the appropriate modelling trade-offs.
by complexity of inference do you mean the complexity of learning the structure of a model ? because inference on an existing PGM is linear in the number of edges with belief-propagation isn't it ?
cycles can be handled in two ways: if you are happy with approximate solutions, loopy BP can give that (still linear, but may take longer and there's parameter tuning), for exact solutions you can rewrite the graph to "carry" dependencies (latest paper by Frey)
PGMs are good if you have knowledge that is important to encode; you need modularity i.e. you don't want to embed priors in the model; you have strong causal relationships e.g. diagnostic models with 'explaining away'; you want to integrate value of information e.g. what test should I do next to resolve uncertainty.
You can mix ML with knowledge in PGMs. The downside is the computational complexity of inference which is NP-hard for both exact and approximate inference, although you can identify the complexity of the model at design time and make the appropriate modelling trade-offs.