One way to understand this is to look directly at the training and inference methods for RBMs.
The simplest training method for Boltzmann machines using gradient descent requires calculating two distributions, one of them being the partition function. As a starting point, we decide to use a Monte Carlo algorithm to sample from it instead and then take some averages to estimate the distribution. Taking the difference gives the direction of the gradient for the weight updates. Sampling can be done by looking at each unit and updating its state (on or off) by calculating the probability of it being on due to the other units' states and how strongly their states are correlated (or anticorrelated) with this one unit. This is a bit of a chicken and egg problem, but eventually if you make enough "sweeps" over all units, you come to some type of equilibrium point.
For fully-connected Boltzmann machines, the number of sweeps it takes to come to equilibrium (or alternatively, the number of steps in the Markov chain) is very large because the dependencies between units can be very complex and it takes many tries to take a statistically significant number of samples to estimate the probability of a unit to be on. This is where the bipartite connectivity of the RBM helps. It requires that we relax the detailed balance condition, but that tends to be an OK assumption so far (nothing can be said for more complicated data modeling). RBM units can be updated by the layer because there are no direct connections between units in each layer. This can make finding equilibrium points easier, but that's partly because we've restricted where these equilibrium points can form through this training process. While RBMs and stacked RBMs in deep belief networks seem to be performing well now, it's not clear that they will continue to do so for more complicated problems.
The hope is that we can do something sort of in between. We want the ability to model complex relationships in data by allowing direct connections between latent variables (the hidden units), but not have so many intralayer connections that convergence of a Markov chain takes forever. The idea is to have a layered bipartite structure in the hidden units. This particular topology has been dubbed deep Boltzmann machines, which differ from deep belief networks in that the entire model is undirected. This is thought to be advantageous for deep Boltzmann machines since information from higher layers can flow to the lower layers, and deep belief networks are already fairly successful at modeling a range of different types of data.
Fully-connected (or flat, as it is called in the paper you've cited) Boltzmann machines can do anything that a Boltzmann machine with sparser graph structure could do, but only in theory. In practice, it takes much longer to train these models and they are almost always unsuccessful with our current training methods. This may have to do with gradient descent being a local optimizer of the likelihood function, but I don't recall any literature specifically exploring this. It certainly has to do with the prior we're enforcing through the restrictions on the network graph. These tricks and approximations are extremely important in practice with our current hardware, because otherwise they are unusable. It is important to make clear that using RBMs in a deep belief network, we can do inference extremely fast because it requires just one matrix-matrix multiplication for each layer. Inference with more complicated networks may require Markov chain sampling too, which would be horrible for performance reasons and we're at a stage right now where performance matters critically for making progress in research, not to mention industrial applications.
------------------------------
Suman Suhag
Dev Bhoomi Uttarakhand University
Data Science And Researcher
+91 8950196825 (Jhajjar, Haryana, India)
------------------------------
Original Message:
Sent: Tue September 16, 2025 11:09 AM
From: JOHN Lundgren
Subject: ANNOUNCEMENT - September 16, 2025 - IBM OpenPages on Cloud adds the ability to integrate with watsonx.governance to add comprehensive AI governance capabilities!
All,
On behalf of the OpenPages Team, I am pleased to announce that IBM OpenPages on Cloud now offers Message Broker providing clients with the ability to integrate with watsonx.governance, adding comprehensive AI governance capabilities!
News Item: https://www.ibm.com/support/pages/node/7245114
------------------------------
John Lundgren
Senior Product Manager
IBM OpenPages and watsonx.governance
jlundgren@us.ibm.com
------------------------------