In food processor bowl with metal blade or blender container, combine cantaloupe and sugar; process with on/off pulses until smooth. Spoon sherbet into processor bowl; process with on/off pulses just until combined. Add buttermilk and ginger; process just until blended.
Pour into 1-1/2-quart freezer container; cover. Freeze 3 to 4 hours or until firm.
To serve, let stand at room temperature for 20 to 30 minutes. Spoon into chilled individual dessert dishes.