Communities are one of the essential elements of modern cities, significantly contributing to overall energy consumption and carbon emissions. To further support ongoing decarbonization initiatives, this study presents a novel Physically Consistent Deep Learning (PCDL)-based Model Predictive Control (MPC) approach for community energy management. This approach incorporates both building-to-grid interactions and on-site renewable energy resources. The PCDL model, starting with the definition of physics consistency, is constructed in line with the established physical laws that govern community thermal dynamics. Serving as a precise thermal load and indoor climate estimator, the PCDL model is then implemented in a centralized MPC to reduce the community energy cost and maintain comfortable indoor environments for buildings in the community. To verify the effectiveness and control performance of the proposed framework, we use a simulation case study of a student residential hall at Cornell University. The results demonstrate that the PCDL-based MPC is highly effective in maintaining comfortable indoor conditions and contributes to load shifting and shaving through its participation in the demand response service.