A detailed analysis is presented in order to determine the sensitivity of the estimated short-term cloud feedback to choices of temperature datasets, sources of top-of-atmosphere (TOA) radiative flux data, and temporal averaging. It is shown that the results of a previous analysis, which suggested a likely positive value for the short-term cloud feedback, depended upon combining radiative fluxes from satellite and reanalysis data when determining the cloud radiative forcing (CRF). These results are contradicted when ΔCRF is derived from NASA's Clouds and Earth's Radiant Energy System (CERES) all-sky and clear-sky measurements over the same period, resulting in a likely negative feedback. The differences between the radiative flux data sources are thus explored, along with the potential problems with each method. Overall, there is little correlation between the changes in the CRF and surface temperatures on these timescales, suggesting that the net effect of clouds varies during this time period quite apart from global temperature changes. Attempts to diagnose long-term cloud feedbacks in this manner are unlikely to be robust.