This research paper presents an innovative model of a programs internal behaviour over a set of test inputs called the probabilistic program dependence graph (PPDG), which facilitates probabilistic analysis and reasoning about uncertain program behaviour particularly that associated with faults. The PPDG construction augments the structural dependences represented by a program dependence graph with estimates of statistical dependences between node states, which are computed from the test set. The PPDG is based on the established framework of probabilistic graphical models which are used widely in a variety of applications. This research paper presents algorithms for constructing PPDGs and applying them to fault diagnosis. The research paper also presents preliminary evidence indicating that a PPDG based fault localization technique compares favourably with existing techniques. The research paper also presents evidence indicating that PPDGs can be useful for fault comprehension.