D&D 5E - Paladin/Bard progression

The next campaign I attend as a player seems to start at L1 and then ends at L5 or so. Thus, I do know that going multiclass is not a great idea. Non the less I am planning to play a Paladin/Bard (College of Swords). Start from a L1 paladin. He is meant to be a member of Harpers and I am planning to make him a good negotiator/spy.

If you were me, how will you develop this multiclass PC? My current plan is,

L1: Paladin (gaining Heavy Armor proficiency)
L2: Paladin 1/Bard1 (for completing this PC's flavor ASAP)
L3: Paladin 1/Bard 2
L4: Paladin 1/Bard 3 (entering the College of Swords)
L5: Paladin 2/Bard 3 (smite)
L6+: Possibly continue leveling up Bard class

Do you think this PC may at least fight so-so at each level? Or shall I re-consider the order or the balance between two classes?

One of my concern is, from L2-L3, he cannnot use his sword as a bard spell focus.