Designing large structures like dams requires carefully selecting various geometric, hydraulic, and structural characteristics. The required structural design and performance criteria are considered when selecting these characteristics. In order to find the best solution, a variety of restrictions must simultaneously be carefully taken into account. This study presents an effective method for determining the optimal shape design for concrete buttress dams. The research was divided into two crucial phases. The dam's initial design and subsequent modeling were mostly done using DIANA FEA and traditional design and stability analysis. After that, a genetic algorithm was used on the MATLAB platform to control optimizing the dam's shape. Three design factors were used in this phase to alter the goal function and to reduce the amount of Concrete used, which decreased project costs. These variables covered three areas of the buttress's cross-section. Two important limitations were scrutinized during this optimization process: establishing a safety margin against overtopping and preventing sliding. The analysis included a detailed assessment of Shear friction stability to complete a thorough stability study. The optimization efforts had a spectacular result, resulting in a significant 52.365% reduction in the total volume of Concrete used, dropping from 19147.5 cubic meters to 9122.55 cubic meters. This decrease was made possible by reducing three distinct components (X1, X2, X3), with respective proportions of 37.5%, 13.33%, and 30%, including two segments related to the buttress and the final segment linked (slab) to the strip footing.