Lots of good inputs here!
Confidence is a funny thing. I can't describe it, but I know it when I see it, and it looks different in different people. Just be yourself and be comfortable in your own skin, and that confidence will shine right through!
Having said that, you can be confident in yourself, love yourself, and not care what others think, and still want to make improvements based on what YOU want for YOURSELF. For example, about 25 years ago, I was married with two young kids, working full time and going to law school full time. I slept about 3 hours per day and picked up some bad lifestyle habits. After law school, I was 5'10" and 235 pounds and carrying about the same amount of lean mass as in high school, where I was only 190. so I had put on almost 50 pounds of fat. Over the next 6-10 years after law school I made lifestyle changes (always helps when your doctor looks at you and says you're going to be dead by 40...) and lost about 50 pounds of fat while putting on about 50 pounds of muscle.
Was that the right thing to do? Yes, it was for ME. My doctor thinks I should have lost 50 pounds of fat and that it, LOL. But I chose to make improvements as I defined that and I'm happy (most days, LOL) with those choices I made. The point of my blabbering is that if you want to lose another 30 pound or so because that's what you want for yourself, then go for it. But don't do that for someone else. If you like the way you look and feel at 170, good for you. Stay put and own it!
If you want to lose bodyfat while holding on to your lean mass, it's doable, but it's typically a slow process (in order to keep the lean mass) and it requires discipline on the diet. I would recommend trimming calories only slightly as you don't want to feel like you're hungry all the time. That's a recipe for failure long term. I recommend a ratio of protein/fat/carbs at about 40/30/30. Having a decent amount of your calories as fat will help you feel more satisfied and less hungry.
Also, to keep the metabolism chugging along, I would recommend you split up all your daily calories into 6 meals instead of 3. Each meal is smaller, but your blood sugar peaks/valleys will be less severe, which prevents the metabolism from slowing. Plus, knowing that you've got another meal to eat at dinner time always helps me to push myself away from the table before feeling "full." I know in another 3 hours I'm going to eat something, so emotionally I'm able to stop.
I don't always eat like this, but if I'm trying to loss fat and keep the muscle, I tend to be more strict like this.
Also, consider implementing some intervals in your cardio. I like HIIT type stuff, but any type of intervals can work.