cntk.contrib.netopt.factorization module

dense_factored(shapes, activation=<cntk.default_options.default_override_or object>, init={'W1': None, 'W2': None}, input_rank=None, map_rank=None, bias=<cntk.default_options.default_override_or object>, init_bias=<cntk.default_options.default_override_or object>, name='')[source]

Perform the new model creation using the factored inputs W1 and W2. The returend function represents the new model.

Parameters:
  • shapes – dimensions of the input matrices.
  • activation – activation function used for the model.
  • init – the two matrices corresponding to the factorization.
  • input_rank – rank of the input tensor.
  • map_rank

    ???

  • bias – bias for the model.
  • init_bias – initial bias value.
  • name – name of the block function that creates the new model.
Returns:

a model that is factored and projected (reduced).

factor_dense(model, projection_function=None, filter_function=None, factor_function=None)[source]

Reduce the size of a dense model using the provided factor_function and the projection_function. filter_function is used to select dense layers to apply the reduction. If no factor_function is specified, use svd decomposition.

Parameters:
  • model – dense model.
  • projection_function – determin the new size of the dense model. It can be based on the shape of the weight matrix or other heuristics. factor_function can choose to ignore the value k.
  • filter_function – filter layers in the model to apply the factorization
  • factor_function – factor the dense model (e.g. svd)
Returns:

a model that is factored and reduced in size.

svd_subprojection(matrix, k)[source]

Calculate svd of the matrix and produce a subprojection based on k

Parameters:
  • matrix – an input matrix
  • k (int) – desired rank of the output matrix
Returns:

two matrices representing the original matrix after svd and reducing them based on k.