No, I haven't read the manga. Yes, in the first few episodes Asta's voice was really annoying. No, it doesn't stay that way.
With that out of the way, this is my review.
The show contains a lot of cliche moments and sometimes things are really predictable. However, they do spend some time in character development and that triggers an interest in what will happen in the future.