Worth of prior knowledge for enhancing deep learning
In deep learning, prior knowledge is essential for mitigating shortcomings of data-driven models. However, there still exist several challenges, including the evaluation of the worth of prior knowledge, the multiobjective optimization of data and rule loss, and the maximization of the effect of knowledge. In this work, we present a framework to enable efficient evaluation of the worth of knowledge quantitatively by the derived rule importance, which deepens the understanding of the nexus between data and knowledge. It is discovered that there exist sophisticated relationships between data and rules, including dependence, synergistic, and substitution effects. Meanwhile, the worth of prior knowledge differs in the in-distribution and out-of-distribution scenarios. The proposed framework can be applied to improve the performance of informed machine learning, as well as to distinguish improper prior knowledge. Experiments have proven that our framework can shed light on diverse fields enc