The Leeb hardness test is widely used for hardness acceptance in the sour oil and gas pipeline construction site. The accuracy of the site test is crucial to the prediction of the risk of pipeline cracking. After reviewing relevant standards and literature, this paper identifies the key challenges in applying Leeb hardness to the acceptance of sour pipeline construction as being the deviation in test result caused by the characteristics of pipeline steel. Subsequently, comparative tests between Leeb and Brinell hardness were conducted on over a dozen pipeline steel materials with varying grades, steel types, and wall thicknesses. The results indicate that significant deviations occur when using Leeb hardness test for both base material and girth weld inspections, with the deviation in results correlated well with the pipe wall thickness. The feasibility of two approaches—pipeline coupling and hardness correction—to mitigate the deviation in Leeb hardness testing is discussed. From the study, it is recommended that Leeb hardness testing in oil and gas pipeline construction should undergo hardness correction to minimize safety risks.