Submodular Functions and Their Applications
Jan Vondrák1
1IBM Almaden Research Center
San Jose, CA
SIAM Discrete Math conference, Minneapolis, MN
June 2014
Jan Vondrák (IBM Almaden) Submodular Functions and Applications 1 / 28
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Guidelines and tips
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Submodular function and extension, Lecture notes of Applied Mathematics
This document is about Submodular functions and their extension
Typology: Lecture notes
2017/2018
1 / 46
This page cannot be seen from the preview
Don't miss anything!
Related documents
Partial preview of the text
Download Submodular function and extension and more Lecture notes Applied Mathematics in PDF only on Docsity!
Submodular Functions and Their Applications
Jan Vondrák^1
(^1) IBM Almaden Research Center San Jose, CA
SIAM Discrete Math conference, Minneapolis, MN June 2014
Discrete optimization
What is a discrete optimization problem? Find a solution S in a finite set of feasible solutions F ⊂ { 0 , 1 }n Maximize/minimize an objective function f (S) Min Cut / Max Cut Min Spanning Tree Max Matching
Some problems are in P: Min Spanning Tree, Max Flow, Min Cut, Max Matching,...
Many problems are NP-hard: Traveling Salesman, Max Clique, Max Cut, Set Cover, Knapsack,...
Continuous optimization
What makes continuous optimization tractable?
A function f : Rn^ → R can be minimized efficiently, if it is convex.
A function f : Rn^ → R can be maximized efficiently, if it is concave.
Continuous optimization
What makes continuous optimization tractable?
A function f : Rn^ → R can be minimized efficiently, if it is convex.
A function f : Rn^ → R can be maximized efficiently, if it is concave.
Discrete analogy? Not so obvious... f is now a set function, or equivalently f : { 0 , 1 }n^ → R.
From concavity to submodularity
Concavity :
f : R → R is concave,
if the derivative f ′(x) is non-increasing in x.
From concavity to submodularity
Concavity :
f : R → R is concave,
if the derivative f ′(x) is non-increasing in x.
Submodularity:
x 1
x 2
f : { 0 , 1 }n^ → R is submodular,
if ∀i, the discrete derivative ∂i f (x) = f (x + ei ) − f (x) is non-increasing in x.
Equivalent definitions
(1) Define the marginal value of element j, fS (j) = f (S ∪ {j}) − f (S).
j
S
T
f is submodular, if ∀S ⊂ T , j ∈/ T :
fS (j) ≥ fT (j).
(2) A function f : 2 [n]^ → R is submodular if for any S, T ,
f (S ∪ T ) + f (S ∩ T ) ≤ f (S) + f (T ).
Where do submodular functions appear?
1. Foundations of combinatorial optimization: [Edmonds, Lovász, Schrijver... 70’s-90’s] rank functions of matroids, polymatroids, matroid intersection, submodular flows, submodular minimization → submodular functions often appear in the background of P-time solvable problems.
Where do submodular functions appear?
1. Foundations of combinatorial optimization: [Edmonds, Lovász, Schrijver... 70’s-90’s] rank functions of matroids, polymatroids, matroid intersection, submodular flows, submodular minimization → submodular functions often appear in the background of P-time solvable problems. 2. Algorithmic game theory: [Lehmann, Lehmann, Nisan, Dobzinski, Papadimitriou, Kempe, Kleinberg, Tardos,... 2000-now] submodular functions model valuation functions of agents with diminishing returns → algorithms and incentive-compatible mechanisms for problems like combinatorial auctions, cost sharing, and marketing on social networks. 3. Machine learning: [Guestrin, Krause, Gupta, Golovin, Bilmes,... 2005-now] submodular functions often appear as objective functions of machine learning tasks such as sensor placement, document summarization or active learning → simple algorithms such as Greedy or Local Search work well.
Outline
(^1) What are submodular functions?
(^2) Is submodularity more like convexity or concavity?
(^3) Continuous relaxations for submodular optimization problems.
(^4) Hardness from symmetric instances.
(^5) Where do we go next...
Submodular = concave or convex?
Argument for concavity: Definition looks more like concavity - non-increasing discrete derivatives. Argument for convexity: Submodularity seems to be more useful for minimization than maximization.
Theorem (Grötschel-Lovász-Schrijver, 1981;
Iwata-Fleischer-Fujishige / Schrijver, 2000)
There is an algorithm that computes the minimum of any submodular function f : { 0 , 1 }n^ → R in poly(n) time (using value queries, f (S) =?).
Convex aspects of submodular functions
Why is it possible to minimize submodular functions?
The combinatorial algorithms are sophisticated... But there is a simple explanation: the Lovász extension.
Concave aspects?
Recall definition: non-increasing discrete derivatives.
x 1
x 2
f : { 0 , 1 }n^ → R is submodular,
if ∀i, the discrete derivative ∂i f (x) = f (x + ei ) − f (x) is non-increasing in x.
Looks like concavity.
Concave aspects?
Recall definition: non-increasing discrete derivatives.
x 1
x 2
f : { 0 , 1 }n^ → R is submodular,
if ∀i, the discrete derivative ∂i f (x) = f (x + ei ) − f (x) is non-increasing in x.
Looks like concavity. But problems involving maximization of submodular functions are typically NP-hard! (Max Cut, Max Coverage, ) So what’s going on?