RSA Signing Link to heading

RFV8017 defines the standard for PKCS RSA which is an update over 1.5 version from 2003. In that spec, RSASSA-PSS algorithm is defines as:

Encoding: converting the message to encoded message
Covert encoded message to octet string
Octet to Integer
Sign Octet with private key
convert signature from int to Octet

This is the copy from the [RFV8017][spec]

8.1.1. Signature Generation Operation

RSASSA-PSS-SIGN (K, M)

Input:

 K        signer's RSA private key
 M        message to be signed, an octet string

Output:

 S        signature, an octet string of length k, where k is the
          length in octets of the RSA modulus n

Steps:

 1.  EMSA-PSS encoding: Apply the EMSA-PSS encoding operation
     (Section 9.1.1) to the message M to produce an encoded message
     EM of length \ceil ((modBits - 1)/8) octets such that the bit
     length of the integer OS2IP (EM) (see Section 4.2) is at most
     modBits - 1, where modBits is the length in bits of the RSA
     modulus n:


        EM = EMSA-PSS-ENCODE (M, modBits - 1).

     Note that the octet length of EM will be one less than k if
     modBits - 1 is divisible by 8 and equal to k otherwise.  If
     the encoding operation outputs "message too long", output
     "message too long" and stop.  If the encoding operation
     outputs "encoding error", output "encoding error" and stop.

 2.  RSA signature:

     a.  Convert the encoded message EM to an integer message
         representative m (see Section 4.2):

            m = OS2IP (EM).

     b.  Apply the RSASP1 signature primitive (Section 5.2.1) to
         the RSA private key K and the message representative m to
         produce an integer signature representative s:

            s = RSASP1 (K, m).

     c.  Convert the signature representative s to a signature S of
         length k octets (see Section 4.1):

            S = I2OSP (s, k).

 3.  Output the signature S.

Python implementation Link to heading

This is the implementation from [github which is extacly the same as the algorithms specified

def sign(private_key, message,
        emsa_pss_encode=emsa_pss.encode,
        hash_class=hashlib.sha1,
        mgf1=mgf.mgf1,
        rnd=default_crypto_random):
    '''Sign message using private_key and the PKCS#1 2.0 RSASSA-PSS
       algorithm.
       private_key - the private key to use
       message - the byte string to sign
       emsa_pss_encode - the encoding to use, default to EMSA-PSS encoding
       hash_class - the hash algorithme to use, default to SHA-1 from the
         Python hashlib package.
       mgf1 - the mask generating function to use, default to MGF1
       rnd - a random number generator to use for the PSS encoding,
       default to a Python SystemRandom instance.
    '''
    mod_bits = private_key.bit_size
    embits = mod_bits - 1
    em = emsa_pss_encode(message, embits, hash_class=hash_class,
            mgf=mgf1, rnd=rnd)
    m = primitives.os2ip(em)
    s = private_key.rsasp1(m)
    return primitives.i2osp(s, private_key.byte_size)

def encode(m, embits, hash_class=hashlib.sha1,
        mgf=mgf.mgf1, salt=None, s_len=None, rnd=default_crypto_random):
    '''Encode a message using the PKCS v2 PSS padding.

       m - the message to encode
       embits - the length of the padded message
       mgf - a masg generating function, default is mgf1 the mask generating
       function proposed in the PKCS#1 v2 standard.
       hash_class - the hash algorithm to use to compute the digest of the
       message, must conform to the hashlib class interface.
       salt - a fixed salt string to use, if None, a random string of length
       s_len is used instead, necessary for tests,
       s_len - the length of the salt string when using a random generator to
       create it, if None the length of the digest is used.
       rnd - the random generator used to compute the salt string

       Return value: the padded message
    '''
    m_hash = hash_class(m).digest()
    h_len = len(m_hash)
    if salt is not None:
        s_len = len(salt)
    else:
        if s_len is None:
            s_len = h_len
        salt = primitives.i2osp(rnd.getrandbits(s_len*8), s_len)
    em_len = primitives.integer_ceil(embits, 8)
    if em_len < len(m_hash) + s_len + 2:
        raise exceptions.EncodingError
    m_prime = (b'\x00' * 8) + m_hash + salt
    h = hash_class(m_prime).digest()
    ps = b'\x00' * (em_len - s_len - h_len - 2)
    db = ps + b'\x01' + salt
    db_mask = mgf(h, em_len - h_len - 1)
    masked_db = primitives.string_xor(db, db_mask)
    octets, bits = (8 * em_len - embits) // 8, (8*em_len-embits) % 8
    # replace first `octets' bytes
    masked_db = (b'\x00' * octets) + masked_db[octets:]
    new_byte = _and_byte(masked_db[octets], 255 >> bits)
    masked_db = masked_db[:octets] + new_byte + masked_db[octets+1:]
    return masked_db + h + b'\xbc'

def i2osp(x, x_len):
    '''Converts the integer x to its big-endian representation of length
       x_len.
    '''
    if x > 256**x_len:
        raise exceptions.IntegerTooLarge
    h = hex(x)[2:]
    if h[-1] == 'L':
        h = h[:-1]
    if len(h) & 1 == 1:
        h = '0%s' % h
    x = binascii.unhexlify(h)
    return b'\x00' * int(x_len-len(x)) + x

def os2ip(x):
    '''Converts the byte string x representing an integer reprented using the
       big-endian convient to an integer.
    '''
    h = binascii.hexlify(x)
    return int(h, 16)

class RsaPrivateKey(object):
    __slots__ = ('n', 'd', 'bit_size', 'byte_size')

    def __init__(self, n, d):
        self.n = n
        self.d = d
        self.bit_size = primitives.integer_bit_size(n)
        self.byte_size = primitives.integer_byte_size(n)

    def __repr__(self):
        return '<RsaPrivateKey n: %d d: %d bit_size: %d>' % (self.n, self.d, self.bit_size)

    def rsadp(self, c):
        if not (0 <= c <= self.n-1):
            raise exceptions.CiphertextRepresentativeOutOfRange
        return primitives._pow(c, self.d, self.n)

    def rsasp1(self, m):
        if not (0 <= m <= self.n-1):
            raise exceptions.MessageRepresentativeOutOfRange
        return self.rsadp(m)