<i>Strongylocentrotus intermedius </i>was introduced to China from Japan in 1989. It is the only sea urchin species cultured in China due to its short spines and high gonad quality. Family selection is being applied to genetically improve growth rates and gonad quality of <i>S. intermedius.</i> Families from different environments (water temperature, stocking density, illumination, and culture facility) rank differently during family selection, suggesting a family by environment interaction. However, the diet by family interaction, which is one of the most important environment factors, has not been examined. In this study, two different diets (artificial feed and <i>Undaria pinnatifida</i>) were fed to six families of <i>S. intermedius </i>to examine the effects of diet, family, and the diet by family interaction on final body weight (FBW), specific growth rate (SGR) for body weight, feed conversion rate (FCR), and change in the coefficient of variation (ΔCV). At the end of the 60-day experiment, we found no significant diet by family interaction effect on FBW or ΔCV. Sea urchins fed <i>U. pinnatifida </i>grew significantly larger and had more uniform FBW than those fed artificial feed (<i>P </i><0.01). Significant differences were detected in both FBW and ΔCV among families. These differences suggest that family selection is an effective genetic method to improve FBW and uniformity. We compared the FCRs of <i>S. intermedius</i> families fed macro-algae and artificial feed for the first time. Significant diet by family interaction effects were detected on both SGR and FCR (<i>P</i><0.01). Sea urchins in all families fed <i>U. pinnatifida </i>had larger SGR and FCR (<i>P</i><0.01). Both SGR and FCR were significantly different among families fed the different diets (<i>P</i><0.05). These differences suggest that both SGR and FCR can be improved genetically using family selection. The families fed the two diets were the same when ranked by SGR, suggesting that the interaction effect on SGR was caused by variations in the families between diets. However, family ranking using FCR was the opposite of that for SGR. Re-ranking the families showed that FCR was different based on diet. In conclusion, our results suggest that growth rates, FCR, and uniformity of <i>S. intermedius</i> were improved genetically by family selection. In addition, diet significantly lowered growth rate, FCR, and uniformity when family selection was applied. Breeders should consider the family by diet interaction when FCR is the target trait.