A galaxy cluster as the most massive gravitationally bound object in the Universe, is dominated by dark matter, which unfortunately can only be investigated through its interaction with the luminous baryons with some simplified assumptions that introduce an un-preferred bias. In this work, we, for the first time, propose a deep learning method based on the U-Net architecture, to directly infer the projected total mass density map from idealized observations of simulated galaxy clusters at multiwavelengths. The model is trained with a large data set of simulated images from clusters of THE THREE HUNDRED PROJECT. Although machine learning (ML) models do not depend on the assumptions of the dynamics of the intracluster medium, our whole method relies on the choice of the physics implemented in the hydrodynamic simulations, which is a limitation of the method. Through different metrics to assess the fidelity of the inferred density map, we show that the predicted total mass distribution is in very good agreement with the true simulated cluster. Therefore, it is not surprising to see the integrated halo mass is almost unbiased, around 1 per cent for the best result from multiview, and the scatter is also very small, basically within 3 per cent. This result suggests that this ML method provides an alternative and more accessible approach to reconstructing the overall matter distribution in galaxy clusters, which can complement the lensing method.