The server can hit the first serve in any direction, with varying amounts of spin, and at any speed they want, on any given serve.
Too often, a player hits the first serve very hard and flat and either out wide or down the “T”. This tends to produce a low percentage of first serves made and forces the server to hit the second much slower. Try using different speeds (start with slow, medium, fast) to give yourself a better first serve percentage and also make it difficult for the returner to time their shot.
It is also a good idea to hit your second serve at different speeds to keep your opponent off balance. Obviously, it is important to make the second serve but altering the speed can make it more effective. Many players use the pace from your serve to add speed to the return. Throwing in a slow, high kick, or a slow, low, slice on either first or second serves will really keep your opponent off balance.
Try to remind yourself to use several different speeds and you will likely start to win points on your serve more often just because of the variety you are giving your opponent.