To remove unwanted followers: tap the 'Me' tab at the bottom bar > 'Followers'. Choose the profile you want to remove and 'Drag the cell' to the left. Tap 'X' and 'Remove'.
This will prevent the person from seeing your activity.
To stop following other people's activity tap the 'Following' list. Tap on the icon with the blue checkmark and then 'Unfollow'.